SBL and methodology…not!
After a long delay, I am now back to writing some more entries. For starters I have some observations on the SBL conference.
First off, I was fairly appalled at the near-total lack of knowledge of even the most basic steps of scientific methodology. While there is no question that most of these PhDs are very knowledgeable and well-read in their fields it became apparent very quickly that their many years of training and study included not even five minutes about the methods involved in formulating a scientifically sound theory. While I am probably unfairly generalizing here, it seemed true in many cases in the lectures I attended. Let’s take some examples:
Waiting for the Kingdom by Giovanni Bazzana. Now, this is obviously a smart and dedicated man. Unfortunately, his theory was far less smart. Actually, to be accurate, there is no way to know if his theory was good, bad, or indifferent. Why? Because he completely neglected one of the most significant pieces of a theory, falsification. I tried pressing him on the issue, asking for some hard numbers. Basically, he has two examples of papyri (PKoeln 7,313, 13-22; and 1 Mac 13, 36-39) where he finds some similarity in the wording between the release of debts and the Q version of the Lord’s prayer. A few natural questions should have occurred to him at this point, they certainly occurred to me right away. Two examples out of how many? He wasn’t quite sure, tens of thousands… Are there any other decrees on papyrus where similar word patterns can be found that match formulaic expressions in the NT? He didn’t know, hadn’t checked. At this point I stopped asking as his eyes were glazing over and he didn’t seem to have the faintest clue as to what I was asking him. I was asking for quantification. He said near the end of his presentation, “I think this is significant,” referring here to his findings. Really? You think it’s significant? Why do I care what you find significant? I don’t! What you think is irrelevant, it is your evidence that needs to convince me. Unfortunately, you showed up without any theory and without any evidence other than two highly dubious textual parallels. If you cannot show some hard numbers, if you cannot provide grounds for falsification, then you have nothing. Well, I guess you do have some interesting musings that might serve as inspiration to others, which is nice but hardly scientific.
Let’s look at another example, this one by Larry Hurtado, which was very disappointing since Hurtado is obviously very smart and has done some truly excellent work over the years. What is even more disappointing is that his work on Mark and Codex W was a solid piece of good scientific work, so he clearly knows how to do these things. I won’t say too much on this topic because Bart Ehrman really laid into Hurtado so effectively that very little remains to be said. Ehrman’s attack was effective and accurate but even he did not truly understand the exact nature of the problem with Hurtado’s work, all he knew was that there was something wrong with it. I asked Hurtado about his conclusions regarding scrolls and codices and their respectives roles, trying to find out how strong his case was. Again, I was trying to get hard numbers. How many scrolls and codices were we talking about, how many had markings to facilitate reading and to what extent? Again, basic information without which it is impossible to arrive at any kind of conclusion. Hurtado seemed to not understand what I was asking. I was beginning to think that I had forgotten how to speak English but subsequent tests proved that I was indeed capable of forming coherent sentences. Ah, well…
Final example, Holger Strutwolf who was talking about text types and the need for a new classification scheme. Now, he may be entirely accurate, again there is no way to know, at least not given the data he presented. He tried to show that the percentage of variation did not group the exemplars into reasonably clear and delineated text type groups. I am curious to know how he measures variations. How does he weigh them? Does he even weigh them? What words are ignored? Why? You can’t just go in and take two texts and count the differences, word by word. At least, not if you expect to get any kind of useful answer. I think he asks an important question but I am far from convinced that he has the knowledge and skills necessary to make this case, one way or another. Check out my previous blog entry on how to weigh differences, How differrent is different?
Enough complaining for now. There were a lot of bright spots as well. It was great to meet Stephen Carlson (again), Jeffrey Gibson, Ken Olson, Catherine Smith and many others. There were some great lectures. I especially enjoyed Thomas J. Kraus speaking about Miniature Books, Codices, or Formats? Categories, Contexts, and Conclusions, Peter M. Head on Named Letter Carriers in the Documentary Papyri, Gilbert van Belle The Use of the Pronomen Abundans in the Fourth Gospel and Kevin Wilkinson on “Hermeneiai” in Manuscripts of John’s Gospel and the Art of Bibliomancy just mention a few.
Anyways, that is it for now. I have been working on a syntactical parser for Koine Greek. It is coming along really, really well and I will be writing about that next time. The problem is that every time I have an oppotunity to sit down in front of the computer, I want to work on my parser rather than writing a blog entry.
TTFN,
Julian
January 10th, 2007 at 3:21 am
Are you aware of Tim Finney’s work at http://gamma.ei.virginia.edu/tfinney/NTText/book/
January 10th, 2007 at 8:46 am
Thanks for pointing this out, Stephen. It is nice to see some research in this direction. I would like to see some better weight asigned to the variation units. I would also be curious to see someone try to solve the variations-over-time problem using a Hidden Markov Model approach. Has this been done already?
Ultimately, Tim Finney will not be overly successful. He will end up with a text that has some not-overly-impressive percentage chance of being identical to the autograph. Statistical research into the NT is very frustrating because it ends up being inconclusive every time. Either way, though, it is good to see some technology being applied to this problem. You should point out your own project in this area, which I think is quite worthy. Did you add minimum edit distance in there yet? Hmm…?
Julian
January 10th, 2007 at 3:25 pm
No, I use “maximum parsimony” with equal weights as the optimality criterion. To assess how well the manuscript relationships are supported by my dataset, I do a bootstrap analysis.
July 2nd, 2007 at 11:07 am
Your point is one that I have often observed myself — I hold a Chemistry degree –, but never seen remarked on elsewhere. I have often wondered, listening to some point about “this parallel proves that theory” whether the idea of trying the same method on some control piece of text has ever occurred to the author.
I have always held the papers of T. C. Skeat on codicology and papyrology in high regard precisely because they are unusual in their attempts to measure what they study and put numbers on them. So it can be done.
I think that you are correct in suggesting that this is not a question of blame for individual scholars, so much as a general problem. Bad scholarship arises from bad methodology which in turn arises from bad example during the PhD years. Some scholars rise above this, and themselves institute checks. The number of scholars who manage this seems to me way in excess of what one might reasonably expect. But others are less fortunate, less gifted, or less careful.
July 2nd, 2007 at 11:10 am
One other point; in some cases the information is simply not readily available. I remember reading an (excellent) paper about chapter divisions in late antique manuscripts, which observed bitterly that the editors of Pliny the Elder had been “fort discret” about whether or not chapter numerals from the volume of summaries appeared in the manuscripts or not.
July 2nd, 2007 at 9:13 pm
Well, Roger. I never thought to see you here.
As for Skeat, I just read a paper by him but now I can’t for the life of me remember what it was about. *blush* Codex A, I think, relating to the Eusebian Canons and whether it was one of the 50 codices ordered by Constantine. It is on my hard disk somewhere, I will have to dig it up. Either way, we can certainly agree that Skeat should be paid attention to.
As for the methodology, I suspect that the problem is somewhat complex (aren’t they always?) but that a few assumptions may be advanced. It seems to me that biblical scholars are generally very learned, i.e. they posses large quantities of knowledge. I would never want to belittle their years of study and their subsequent proficiency in their respective fields. However, it would appear that biblical scholars are motivated by their faith and thus are driven to learn what one might term maximum quanta of knowledge. Methodology is largely ignored because they are not really scientists, nor is their audience. Falsifiability doesn’t play a role in faith and thus never enter into their minds with regards to their research. One very notable exception here would be Michael Palmer’s Levels of Constituent Structure in New Testament Greek, the only problem with his study is that his conclusions are obvious and were quite well-known long before his book. It is also short on detail but I digress. Excellent work nonetheless in the scientific sense. As for biblical scholars, their knowledge is fine, its application frequently lacking. As I mention above, there are exceptions to this like Hurtado’s Codex W dissertation, Carlson’s stemmatics, and Wilker’s PCA. Good knowledge coupled with sound methodology. As most physicists will attest to, until one can measure something and assign a number, one really doesn’t know what it is or how much.
Julian
July 14th, 2007 at 4:45 pm
I cannot say that I perceive biblical scholars as faith-driven; on the contrary, if anything, they are terrified of being so perceived. But there might be legitimate differing opinions on this, of course. The learning is certainly real, of course.
Have you got any further with the Greek parser?
July 16th, 2007 at 7:58 pm
There are as many approaches as there are scholars, a truism in any field, to be sure. In this particular field things tend to be rather fuzzy, we rarely have evidence in the same sense that the mathematician, astronomer, or even chemist (
) has it. I would like to see a greater focus on statistical approaches and reasonable propositions of falsification on statistical grounds. It would be a useful start and any faith bias could be marginalized on the science side. I think that this can only be taught by example, at least until the academic institutions catch up, and probably by the younger generation of scholars.
This leads me gracefully on to my next point (younger scholars, that is), namely the status of my parser. I haven’t worked on it for a few months so I am trying to assess where I left off. Either way, it is a large question, well, a large answer, actually. For this reason I have decided to write a proper blog entry on the topic as soon as I figure out where everything stands.
In the meantime I can say that I have been talking to Matthew O’Donnell (and Cat
)about about joining forces with them over at http://www.opentext.org and see what our combined efforts can bring about. The problem is that Matt is very busy and so am I. *sigh* It takes months between our e-mail exchanges. And now he (and Stanley Porter and Cynthia Westfall) have started the Linguistic Institute of Ancient and Biblical Greek (http://www.liabg.org/), including a refereed journal and a symposium. I have been invited to the latter and, if I have time to go, maybe we can get that cooperative effort going a bit faster. Either way, once I get a bit further along I will be freely sharing the source code and research with any who want it.
Julian
July 19th, 2007 at 9:20 pm
Okay, I actually got around to it. Here is a new page on the parser topic: http://www.julian.textcrit.com/?page_id=13
Julian
March 3rd, 2009 at 7:17 am
Hello webmaster
I would like to share with you a link to your site
write me here preonrelt@mail.ru