I want to take a step back every so often from the day to day work I do on Literal in order to reflect on what got done during the week and evaluate the direction I want to move things. On one hand this furthers the open source approach I'm taking with the project in general, but on the other hand at this early stage, the primary function of these posts will be as a place for me to log progress. In this, I am clearly inspired by the similar Pioneer framework, and though I have no immediate plans to participate, I think the practice of weekly "updates" is a beneficial one to adopt for myself. I intend for these to be written in a relatively informal voice, relatively quick to write, and cover both the technical and non-technical aspects of product development.

Context

First, as this is my first time writing about Literal, it's worth establishing some context. Literal is a textual annotation management system. Literal is not a general purpose note-taking application, and instead is specifically designed for capturing, organizing, and contextualizing textual annotations.

As Literal is a side-project that scratches a very personal itch, my personal user story is as follows:

1. I am reading text on my phone. In the "time before COVID-19", this usually took the form of reading a PDF document (currently, Braudel's "Civilization and Capitalism") during my 45 minute subway commute to and from my real job. Lately, this has instead taken the form of reading on my phone in bed.
2. A sentence or paragraph is important to me, so I highlight it.
3. I take a screenshot of the highlight and use the Android share action menu to share it to Literal. Literal parses the text from the image and creates an annotation. I can further organize and contextualize the annotation by tagging it.
4. Later, I can search for a specific annotation by text or source, or browse related ideas (i.e. annotations) across contexts (i.e. sources) through tags.

I will spend more time elaborating goals and non-goals of the project as I further develop it — the project itself should serve as the manifesto — but hopefully this user story serves as useful context while the house is still being built. Additionally, the landing page and README have more words. Finally, it's worth mentioning that Literal is strictly a personal side project, and that the pace of development reflects that. Currently, I'm lucky if I get in 15 hours per week of work in across mornings and weekends. In the coming weeks I will transition out of my current role in favor of part-time contract work (get in touch!) and will have more time to spend on Literal. With this context in mind, my notes and thoughts for the week follow.

New Annotation Flow

The first flow I built into Literal was the flow for creating an annotation via the Android share action. This handles what I imagine to be the most common scenario, where a user is reading text in an external application and wants to record it into Literal. This week, I extended the functionality so that there's also a flow for creating new annotations from within Literal:

Tag Transactions

I am currently working on the tagging system for annotations. An annotation (i.e. the highlighted piece of text from a source document) has a many to many relationship with tags. Tags are not a new user experience and Literal's initial version does not re-invent any significant wheels. I'm still in the weeds with backend and data plumbing, but expect some screenshots of the flow in the next week or so.

Literal is built with AWS Amplify. It's my second time using Amplify (trashed, a previous side-project, being the first), and I have many thoughts about it that I'll save for a dedicated post, but the primary thing to be aware of here is how Amplify treats the API and database. Amplify exposes a GraphQL API for your front-end application. As the developer, you write the GraphQL SDL that defines the type, shape, and access patterns of data that you want to expose in your API, and Amplify automatically generates the required set of GraphQL resolvers for that API against DynamoDB. Literal's SDL can be seen here.

The vast majority of the time, this framework works without intervention, e.g. I can add a new top level model type to the GraphQL SDL, and Amplify generates the required set of permissioned CRUD resolvers along with all the CloudFormation templates necessary for the DynamoDB table, AppSync configuration, IAM policies, etc. In particular cases, as was the case this week with tags, I have to pull the curtain back and write some of this logic myself.

The many to many relationship associating tags with annotations involves managing 3 different DynamoDB tables - Tag, Highlight, and HighlightTag ^[1], the join table in the middle. I wanted to be sure that the relationships between the tables was properly constrained as records were added and removed as the result of user-level edits.

DynamoDB supports write transactions, but there's currently no "native" support for leveraging the functionality within Amplify. To add a GraphQL mutation backed by a TransactWriteItems to the appropriate DynamoDB tables, I had to do the following:

1. Add the Mutation field to the SDL.
2. Add CloudFormation resources for the custom AppSync resolver and IAM policy to grant access to all of the required tables.
3. Add the AppSync resolver request and response templates, which do the bulk of the work in getting GraphQL and DynamoDB to speak to eachother.

I did similar work for a couple of other custom mutation fields I required. It's important to recognize that "pulling back the curtain" or "unwrapping the black box" is even possible in Amplify. It's a little painful to be sure (Automatically generated CloudFormation templates? A restricted template language for business logic?) but it's possible. Highly managed services like Amplify commonly lock the developer into a particular level of abstraction that is completely opaque - Firebase is an example of such a service.

Wrapping Up

That's all for this week. Next week, I hope to share some screenshots of what the application currently looks like, handling DynamoDB change streams in Lambda, and the beginning of the tags as contexts user experience.