Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Contributing to TanakhCC

Thank you for your willingness to contribute to TanakhCC. This project is only possible because people just like you invest their time and expertise into making Gods word freely available to all.

This guide will tell you everything you need to know to get started contributing to TanakhCC and requires no previous experience in working with digital transcription. If you just want the raw technical details, check out the Critic TEI Specification.

To give you an overview of the material we will cover here, the following is a birds eye view of how work on TanakhCC works:

  • At the core of our workflow, there is a github organization which hosts the actual raw data, both the current master version as well as all older historical versions of the transcription.
    • tanakhcc.org pulls data from the master branch daily to provide the data in a form searchable directly on the web
  • Transcriptions are stored in xml format, following our own subspecification, compliant with the TEI specification. Any software that can read TEI data can immediately import transcriptions from TanakhCC.
  • To make transcribing more accessible, critic.tanakhcc.org exists as a user interface to allow even non-specialists to easily transcribe manuscripts or reconcile transcriptions.

Here are the steps that happen until a part of a manuscript becomes available in TanakhCC:

  • A project maintainer adds images of the manuscript together with metadata on critic.tanakhcc.org.
  • Two or more project members transcribe this manuscript independently. When they feel their transcription is finished, they can publish it, making it visible to all other project members.
  • A project member takes a manuscript page for which at least two transcriptions have already been done and reconciles these transcriptions. In this step, they decide how to handle complicated cases in the transcription where the transcribers have different opinions on how to represent a text.
  • Once the recollection is done, a merge request will automatically be created on github. This notifies all transcribers and reconcilers involved in this page and includes a report of choices taken during the reconciliation step, based on which a project maintainer will decide to give a final thumbs-up.
  • The page image has now been viewed by at least four different people: two transcribers, a reconciler, and a project maintainer and is merged into the master branch. One day later, the new data will be shown to anyone visiting tanakkhcc.org/read .

Contribute to the guide

Something was unclear, there are typos or you have a suggestion to improve this guide? Open an issue or a pull request on github.

License

This guide is licensed as CC-0. Feel free to reuse it in any way you want.

Transcribing

What makes a good transcription?

TODO - reconstruct the original NO EDITORIAL OPINION IN TRANSCRIPTIONS

How to start?

First of, head over to critic.tanakhcc.org. You will be redirected to a login page where you need to create an account when you first visit. Create an account or log in to use critic.

You can find manuscript pages to transcribe in a neat list in critic. You will find some stats about the page in question, such as the contained verses, or how many transcriptions have already been started and finished for this page. Simply click on the page row and you will be taken to the transcription editor and can check out the page image itself.

Here is how you can select a page that is right for you to get started: Look at a few ones (perhaps choose one with your favorite verse to start), choose one that looks good to you and where you do not have much trouble reading the handwriting. If you choose a manuscript that already has one or two other transcriptions, that increases the chance that your transcription might become part of TanakhCCs official data quite quickly. Seeing your work pay off is motivating, so consider choosing manuscripts someone else has already transcribed - especially for your first few pages.

Using the Transcription Editor

(TODO:) Link to github issue exists, which automatically starts the issue with the correct links and tags.

TODO - big section What you will do here Shortcuts Go over all the different blocks Go over semantics while we do this - will mirror content from the TEI spec

Publish your transcription

Once you are satisifed with your transcription, publish it by pressing the ... publish button at the bottom right of the transcription editor. This will allow others to see your transcription when reconciling and moves your work into the next phase of preparation before it will join TanakhCC.

Now you have already transcribed your first page. Thank you for your hard work, your time and effort spent contributing to TanakhCC. Your dedication helps provide free access to Gods word - accurately and in a modern format.

You may continue by transcribing another page right now! Once you have completed (TODO - how many???) transcriptions, you will become eligible to also reconcile transcriptions, which is described later in this guide.

Transcription Manual

Transcribing in XML

If you are familiar with editing XML, and feel more comfortable using a normal text editor, you can do all transcription work in raw XML.

Semantically, the same rules and guidelines from the guide for manual transcriptions apply.

You can find the Relax-NG schema that files have to adhere to here. I suggest you use a Relax-NG extension for your text editor of choice and automatically check validity while you type. Please note that critic imposes some additional semantic restrictions that are not represented in our schema. To find out exactly how a transcriptions XML has to look like, check out any transcription on github, and make sure to read critics subspec definition.

[!NOTE] Note in particular that metadata and public domain dedication have to be present for every individual page of a manuscript. You should copy the document with its empty body from critics transcription editor to have a quick and correct starting point.

Once you have created your transcription, simply copy-paste it into the XML field in critics transcription editor. If any problem occur parsing your XML file, you will be notified. Otherwise, you will find the structure represented in the other views once parsing completed successfully.

That's it. Happy transcribing!

Reconciling

Once you have completed (TODO - how many???) page transcriptions, you may also reconcile transcriptions. Just like scribes can make mistakes copying written text, transcribers can make mistakes digitizing it.

As a reconciler, you will take two or three transcriptions and compare them to what you see on the page. When the two transcribers differ in their opinion of how the manuscript is best represented, you will decide between the different versions.

Here is how the recociliation workflow looks like:

  • Go to the reconciliation todo-list. You will be shown only pages that have no reconciliation in progress, with at least two transcriptions published.
  • Select any page that you like. You will be taken to the reconciliation editor. Please be patient here - the different transcriptions are not compared to each other and laid out in your browser, which can take a second.
  • Next, you get a side-by-side view of the different transcriptions. Simply select between the different options when transcribers had different opinions about a passage.
    • If you feel like a more complicated combination of the options provided by the transcribers is required, you should select all the different versions you may want to include. You can edit the final reconciliation later.
  • When you are done, click Continue. You will now drop into something like the transcription editor, with your choice of transcriptions made in the last step as a starting point.
  • If you do not feel like any adjustments need to be made, you are done. Do not overcomplicate it. In most cases, the transcribers did a great job, and you will already have a perfectly valid reconciliation.
  • If you feel like adjustments are required, you can edit the reconciliation like you would any normal transcription.
    • Only do this if the available transcriptions do not have the correct reconstruction among them. But: if an edit is required, make it.
    • When you manually edit a reconciliation, the merge request may take longer to approve. Please be patient when project maintainers may need to look closer at the transcription before merging to TanakhCC.
    • And remember: If you ever have any questions, feel free to ask them as a github issue. (TODO:) You will find a direct link to ask a question linked to this page and the used transcriptions directly in the reconciliation editor.
  • At the end, very carefully read over the reconciliation side-by-side with the manuscript image one last time so you have not missed anything. This may feel like a chore, but this additional ensure we do not miss anything. You will probably be the third or fourth, and usually the last person to look at the transcription word for wird before it gets merged into the master branch, so you make sure that you can put your name behind its accuracy.
  • Then, click Request Merge Now. Your Reconciliation will be passed to one more person for a final quick review before being merged into TanakhCC. You can check the progress of this review, and will be shown a link to do so after you created the merge request.

Project Administration

When to approve Reconciliations

Please always be gracious for the effor of transcribers and reconcilers who volunteer their time. Seriously listen to their opinions and take time to write friendly and understandable replies. Your job when approving merge requests for reconciliations is not to retranscribe the page, or to take a lot of your time to rething problems that 3 or more people have tackeled before. Your goals during the approving of a merge request are:

  • Catch gross errors. These can be assumed to not exist in places without differences between transcriptions.
  • Answer uncertainties of transcribers and reconcilers.
  • Decide in situations where opinions of transcribers and reconcilers is closely (or even 1:1) set against each other.

If you do not see obvious problems with the proposed transcription, there are no open questions about the page being merged, simply scan through places where there is dissent between transcribers and the reconciler. Decide in these questionable cases, ask if anyone feels the need to discuss your choice and close the merge request a few days later. To retain volunteer motivation, it is important to keep the feedback cycle quick.

When needed, changes to the proposed final XML can be made directly in github (or copy-paste the data into your own text editor and change it there). When finished, add the change as a new commit in the merge request.

  • If the reconciler has chosen the majority view amongst transcribers, only seek clarification if you have serious doubts about that reconstruction.
  • If the reconciler chose a minority view or the split between transcribers was even, check that passage for yourself. Err on the side of the reconciler in that case - they already spent time sifting through the same problem you are looking at right now.
  • If the reconciler chose a completely unique solution to a passage, please carefully check it against the pages original image. Err on the side of the transcribers, if a majority exists there.

If you are also uncertain how to deal with a passage, feel free to tag more project maintainers and get another opinion.

TODO - Adding MSs

You can add Manuscripts via the Administer page of critic.

TODO - About Page Image Copyright

It is VERY important that all images uploaded to critic may actually be used in that way. Make sure that the copyright on the image permits our use of it, and attribute the image source if required by the license.

TODO - How Versification schemes are reconciled

The critic TEI subspec

![NOTE] The definitive rolling version of this file is the one hosted as part of critic-format, the tool used internally by critic to parse XML. The version here will be updated when changes are made in critic-format.

This Document

This document is written as a guideline for the creation and use of TEI documents employed in critic. Some aspects of this document refer not specifically to TEI encoding, but broader aspects of critic, like its file structure.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

File Layout

Each page (not folio) of an ancient manuscript MUST reside in a separate TEI file.

File Naming Scheme

The names for TEI files SHOULD be {manuscript name}_{folio}_{recto/verso} for folios.

Manuscripts in scroll form (or other forms without natural pagebreaks) SHOULD be given as an individual TEI file for each image. Their filename SHOULD be {manuscript name}_{image-id}

Directory Structure

critic uses this directory structure. For interchange purposes, you are free to use any directory structure that seems convenient.

  • {manuscript name}
    • page files

Metadata

The TEI Header

This TEI Header MUST be present in every individual TEI file.

titleStmt

The title inside titleStmt MUST be given as {manuscript name} folio {folio number} {recto/verse}. when the manuscript has folios or {manuscript name} page {page number} if it does not (e.g. for scrolls).

publicationStmt

The publicationStmt MUST be given as <p>This digital reproduction is published as part of TanakhCC and licensed as https://creativecommons.org/publicdomain/zero/1.0.</p>. In particular, all data that is part of TanakhCC MUST always explicitly be CC0.

sourceDesc

The sourceDesc MUST be given and contain exactly one element msDesc, defined in the next subsection.

Manuscript Description

  • The Body of each file MUST begin with a msDesc.
  • The msDesc MUST have
    • msIdentifier, defining the physical manuscript reproduced:
      • msName MUST be given
      • idno MUST be given and MUST be either {folio number} {recto/verso} or {page number}
      • institution SHOULD be given if relevant
      • collection MAY be given if relevant
  • physDesc MUST be given and contain handDesc and scriptDesc to describe the characteristics of scribal hands or the script used.

Representing the Text itself

As a general rule of thumb, our goal is to reproduce the physical text as closely as possible. If a reconstruction is not obvious, it is preferable to skip it; otherwise we risk corrupting the datapool with reconstructions from other text types. Reconstructions SHOULD be done in the following cases:

  • When characters are difficult but possible to read. Otherwise, reconstructions MAY be done, but SHOULD be avoided, unless their certainty is high. Reconstructions MUST NOT be done to correct scribal errors, including but not limited to:
  • incorrect orthography
  • homoioteleuta / -arkta
  • skipped words, even when alliterations or other markers make a genuine mistake probable All of the above phenomena MUST be represented without emandation. See the later sections for the correct markup to use.

Defining the source Language

  • The source language MUST be defined for each part of the transcription.
  • The source language MUST be defined in the xml:lang attribute.
  • The source language SHOULD be defined on the highest level possible, up to the body tag.
    • When there are different source languages given at different depths, the first language encountered when traversing the manuscript upwards is used.
  • BCP47 Codes MUST be used to define the source language.
    • Common Codes used for the transcription of biblical manuscripts are:
      • hbo-Hebr, hbo-Phnx, smp-Hebr, grc

Text Structure

Text is structured as two levels of nested divs:

  • column (represented by a div with type="column")
    • line (represented by a div with type="line")
    • Every block of normal text MUST be enclosed in <p> unless it is special in some way (see below for special cases). Notice that one page is one OS file, so no folio/page-breaks will ever occur inside one file. For both column and line, the n attribute with the correct number SHOULD be given. If it is not given, the number is supplied sequentially, starting from 1 When lines or columns are missing, n MUST be applied.

Versification

It is the goal of text criticism to be as faithful to each manuscript as possible, and to approach ancient texts on their own merit. Versification is very convenient for everyday use, but it is my opinion that faithfulness to the witness beats convenience. Therefore, physical structure (layout in columns and lines) is given priority over logical structure (versification). Manuscripts that have verse breaks (particularly: sof passuqim in MT manuscripts) SHOULD be transcribed - they are physically extent artefacts, so we want to aknowledge them.

Whenever verse breaks are:

  • physically apperent (sof passuqim, line indentation, actual numbering, ...) OR
  • reconstructable with certainty (comparison with other MSS or standard versions) then the beginnings of verses SHOULD be indicated by <anchor> tags. The <anchor> tag...
  • MUST include type="{the-versification-scheme}".
  • MUST have xml:id="A_V_{the-versification-scheme-shorthand}_{the-verse-ID}".

Considerations when adding Verse anchors

Verse anchors are used to align texts in the collation phase. This means that wrong verse anchors will lead to wrong collation and all sorts of Problems. The strict rule is therefore: When Verse boundaries are uncertain (e.g. words moving between two adjacent verses in different versions), verse anchors MUST NOT be used.

On the other hand, when different versions have text rearranged or differences in verse numbers and the verse boundaries are certain, it is RECOMMENDED that verse anchors be supplied in as many versions as possible. This aids collation and indexing of the data.

Available Versification schemes

In ALL cases, the abbreviation for books from OSIS MUST be used. While this may use book abbreviations foreign to some(or most) schemes, this simplifies crossreferencing drasticly.

Verse IDs in all schemes are formed as: {book_abbr}-{chapter}-{verse}

Common

Shorthand: C Long Form: Common

When all or most versification schemes aggree, you may choose to use the Common scheme instead of entering the same verse number in all the different schemes. This scheme SHOULD be used, when all of the following schemes agree:

  • Masoretic
  • Septuagint
  • ESV
  • Present When the above schemes do not agree, the Common scheme MUST NOT be used. If other schemes disagree on the verse boundary in question, their anchors MAY be given, but it is NOT RECOMMENDED to do so. Instead, disagreement for uncommon versification schemes is best noted in a manuscript for which the uncommon scheme is the natural versification scheme. By way of example: when Aleppo disagrees with BHS, a septuagintal manuscript SHOULD NOT add an anchor for the Aleppo versification.
Present

Shorthand: P Long Form: Present

The verse numbering (explicit or implicit, by counting from the chapter starts) used in the present manuscript.

Masoretic (BHS)

Shorthand: MT Long Form: Masoretic

The Mainline Masoretic Versification Scheme is that used by the BHS.

Masoretic (Aleppo)

Shorthand: MT-Aleppo Long Form: Masoretic-Aleppo

The versification used in the Aleppo Codex.

Septuagint (???)

TODO: prb Rahlfs???

Vulgata
Samaritan Pentateuch
English Standard Version

Shorthand: ESV Long Form: ESV

The versification used in the ESV, 2016 Edition. This Versification scheme is included, since it is commonly used by modern sources.

Abbreviations

When a sequence abbreviates another sequence:

  • The entire part of text MUST be enclosed in <p><choice>
  • The surface form MUST be given in <abbr>
  • The (conjectured) expanded form MUST be given in <expan>

Ligatures

TODO: just use gaiji to represent them as glyphs. Use abbreviation as before for NS and the like

Damaged Characters

This occurs where characters are present, but physically damaged.

  • When a reconstruction is too difficult to make with any certainty, use Lacunous Elements instead.
  • This section does NOT apply when characters are impossible to decipher from their remaining artefacts alone. Reconstructions based on grammar, common words or phrases alone MUST NOT be made. Use Lacunous Elements instead.

The damaged characters MUST be enclosed in <p><damaged>.

  • @agent SHOULD be given
    • good examples of reasons include: smeared, burned, water
    • the @agent MAY NOT contain whitespace
    • If a space would be desired to split words (e.g. smeared diacritica), - SHOULD be used instead (smeared-diacritica)
  • @cert SHOULD be given

Lacunous Elements

This occurs when the existance of characters, lines, columns etc is obvious (for grammatical-, layout- or other reasons), but they are illegible. Whether physical traces of these characters such as illegible smears of ink are present or not is irrelevant. If the characters are hard but possible to read, use Damaged Characters instead.

Instead of missing characters, a <gap> element MUST be used, directly inside the lines <div>.

  • @reason MUST be given
    • good examples of reasons include: lost, smeared, burned, water
  • @unit MUST be given and MUST be one of character, line, column
  • @n MUST be given
  • @cert MAY be given and qualifies both the certainty in assertaining the amount of missing units as well as the proposed reconstruction if any

Ancient Corrections

When multiple ancient surface forms are present in a place, these rules apply. Versions are here considered in (conjectured) temporal order. So when a word was written by Scribe1, struck through by Scribe2 and then another word written atop it by Scribe3, there are three distinct versions.

  • All different ancient versions (the first, and all different corrections) MUST be given separately.
  • The entire passage in question MUST be wrapped in <app>.
  • Every individual version MUST be given in <rdg>.
  • @hand SHOULD be given for all <rdg>s.
  • @varSeq MUST be given for all <rdg>s and mark them in (conjectured) temporal order. @varSeq MUST NOT contain additional semantic information. Normalization MAY turn it into zero-based indices.
  • The text in <rdg> MUST be normal text without additional markup elements.

If part of the text in any of the versions is lacunous, exclude it from its reading. Instead, add a lacuna after the entire correction. If part of the text in any of the versions is damaged but legible, simply add it without markup.

Significant space

When a significant space is left in the ancient manuscript, this MUST be recorded using the <space> element.

  • normal spacing between words MUST NOT be recorded
  • space MUST be recorded if it is both (1) large enough to fit two full-width-characters (i.e. ignoring punctuation or י) and (2) the normal word-spacing of the manuscript is smaller then this space.
  • space MUST be recorded if it spans an entire line or more
  • the @quantity MUST be given
  • @unit MUST be given and MUST be one of character, line, column

Nonstandard Glyphs and Diacritica

Non-Tiberian Niqud

For vocalization that is not tiberian, you MUST transcribe the vowels with the equivalent tiberian niqud. An enclosing element (see above, Source Language) must carry the language tag with the following tags:

  • hbo-Hebr-x-babli-e
  • hbo-Hebr-x-babli-k
  • hbo-Hebr-x-palest