Documentation

The migration process involves a plugin located at that takes a book in HTML format and automates most of the publication process.

Before start

You will need the HTML file for the book from NYU Press.

You’ll need Github access to the repository holding the contents of the website.

And, due to limitations on how Github can allow remote access to repositories, you’ll need a “Personal Access Token” (PAT), which a time-limited password to give other services access to your Github account.

How to get a PAT

Go to the tokens (classic) section on your Settings. To navigate there, use the link provided, or go through Settings > Developer Settings

Personal access tokens > Tokens (classic).

Generate a new token, you will be asked for a description (1), which can be anything you want, and then an expiration date (2) and a list of “scopes” (3).

Set the expiration date to whatever you feel comfortable. You’ll have to get a new token after the expiration date to get access to the NYU Panel.

The only “scope” (permission) needed is “repo” (full control over repositories).

Once you click “Generate token”, you’ll be show the token once. Save it carefully (for instance, on a password manager).

If you lose it, or believe someone could have malicious access to it, you can come back to this section to revoke it.

Adding the book through the panel

Navigate to https://nyu-dss.sutty.nl and use your PAT to access.

Pick the repository for the website you need to add the book to.

You’ll find a list of previously published books and a “create book” section (1). When you open it, you can add a title (2), attach the HTML file for the book (3). When you click the “Create book” button, the migration process will start.

You’ll see a message asking you to wait for a few minutes until the process is done, with a link to Github where you can follow the process.

You can reload this page after a while to find the new book on the list.

Adding a new edition to an existing book

The process is the same as adding a new book, but the file attachment step is done by opening the book from the list.

Under the hood

What just happened is a semi-automated version of the manual steps outlined below. When you upload the HTML file, a “Github Action” runs, extracts all essays, works cited, authors, etc from the file and creates the necessary files to represent them on the website.

A “Pull request” (PR) is created on the Github repository. Once you’re happy will all the changes to the website done on this PR, you can accept it by merging it.

The merge will add the changes to the main branch, which will then be published on the main website.

Adding the book from Github

• Navigate to https://github.com/nyu-dss/keywords-data. You’ll need an account with access to this repository.

• Make sure you are on the main branch.

  

• Go to “Add file” and choose the “Create new file” option.

• Create an about page inside the migration directory.

  On the “Name your file” field, enter _html/, when you press the / it will create a new field, enter the slug here and add a new /.

  Now add about.md as file name and paste the contents on the text box. The first paragraph is going to be the excerpt on the book page.

  If you don’t have an about page yet, create an empty .keep file which is a convention for making Git/Github keep empty directories.

  

• Scroll to bottom to the “Commit new file” box. Add a title and optional description and select the “Create a new branch” option.

  When it asks for the branch name you can leave the suggestion or use the slug for consistency.

• You will be sent to an “Open a pull request” page. You can leave this page open an open https://github.com/nyu-dss/keywords-data in a new tab.

  

• In this new tab, select the newly created branch to switch into it.

  

• On the file list, open _html/ and then the newly created directory. You’ll see the .keep file.

• Now go to “Add file” and choose “Upload files” and upload the HTML file. On the “Commit changes” box add a title and optional description, and leave the “Commit directly to XXX branch” option checked.

  

• Go back to the “Open a pull request” tab or use the suggested “Compare & pull request” button to open a PR.

• Opening the PR will start the migration process.

Adding the book from git command line

• Create a new branch from main using the book slug and switch to it:

  git checkout main
  git checkout -b gender-and-sexuality-studies
  

  Note: The branch name doesn’t matter, it’s just for consistency.

• Create a folder with the slug inside the _html/ folder.

  For instance, “Gender and Sexuality Studies” has a “gender-and-sexuality-studies” slug.

  You’ll have a _html/gender-and-sexuality-studies/ folder.

• Place the HTML file inside this folder and rename it to index.html.

  You’ll have a _html/gender-and-sexuality-studies/index.html file.

• Commit the changes and push to the repository.

  git add _html/gender-and-sexuality-studies/index.html
  git commit -m "Gender and Sexuality Studies"
  git push -u origin gender-and-sexuality-studies
  

• Go to the Github web interface and create a Pull Request.

After the book is added

Link works cited

Through the panel, you can also link works cited withing essays.

When you open a book after uploading it or even if it was uploaded already, you’ll find a list of essays from that book.

When you open an essay, you’ll find a two pane screen with the essay contents on the left (1), and a couple buttons and a search bar on the right (2).

The panel does its best to find what looks like citations within the essay, and marks them for you (3). If you click on them, the search bar (4) will fill out and provide results (5). If the search results are not accurate, you can change the search terms, until you find the work cited. Use the “Apply” button to link the work cited to its citation.

You can ignore citations that are not correct.

If a citation wasn’t found automatically, you can select the text and use the “Add work cited” button (6) to add it.

Click “Save changes” to save changes.

When you finish with an essay, go back on your browser to continue with the next one.

Review manual actions

When you open the Pull Request (PR for short), a Github Action drops in and runs the migration process. If everything goes OK, in a few minutes you’ll find a new commit on the same PR with all the pages created.

The OK is a green checkmark on the right of the commit message. If it failed, you’ll see a red X mark. Pressing it opens the details. Show should check the details, the migration process can’t do everything automatically and will require manual intervention in a few cases. It will tell you what action you should take.

Read below for messages and what they mean.

Once changes are done, you can merge the PR into the main branch. This will run the build and publication process, at the end of which the site will be live at https://keywords.nyupress.org/

Regular messages

• Creating Gender and Sexuality Studies, don’t forget to add color to _posts/2021-01-01-gender-and-sexuality-studies.markdown

  This message reminds you to add the color attribute on the book file.

• Couldn’t find Acknowledgments

  This message is looking for certain pages to exist on the HTML file. If the book doesn’t have this section you can ignore this message. If the book does have it, it probably means the internal structure for the HTML changed and the plugin needs to be updated.

• Prevented empty post name for # (_posts/2021-01-01-gender-and-sexuality-studies-35.markdown). This could mean the title is empty or composed of non-alphabetic characters.

  Sometimes an essay title can’t be slugified because it only contains non-alphabetic characters. In these cases, to be able to create the file with a consistent name, the migration process creates an ad hoc slug composed of the book’s slug and the ASCII codes for the characters (35 is the ASCII code for #).

  The slug on the file name isn’t really important, but you should edit the permalink field so it has an URL. If you don’t do this, the URL will be the same as the ad hoc slug.

• Couldn’t detect first and last names for this person, please fix manually: (_posts/2021-01-01-hershini-bhana-young.markdown)

  This means the person has more than two names and the migration plugin can’t guess which of the middle words are first or last names (and it shouldn’t).

  This affects the sorting order on the Contributors page. To fix this, edit the list files manually and add a first_name and last_name (specially the latter) field on each one.

• Creating person profile who isn’t linked to any page or essay: Anisha Ahuja

  The person is listed on the Contributors page but didn’t author an essay or isn’t an editor. The migration process creates a profile and links it to the book, so you shouldn’t have to do anything. But if the person is an author/editor, this may mean the HTML file is missing authorship information.

Rare messages

• Found similar name for …

  Sometimes authors/editors and contributors are spelled different (a middle name or initials added, etc.) so the migration looks for similar names. If it didn’t find a correct match, you need to fix this manually, by creating a person profile before redoing the migration.

• Couldn’t find SECTION

  This means the HTML structure changed and the migration plugin needs to be updated.

• Couldn’t process work cited …

  The work cited format couldn’t be detected and the migration plugin couldn’t find an authorship part.