I'd love some early feedback on Margin, a markup language for hierarchically structured thought.
Margin came out of my desire to build a lightweight to-do app – though I soon realized what I really wanted was a markup language to capture structured thought. One that was not only human-readable, but also easily machine parsable.
Most importantly, Margin doesn't impose strict hierarchical categories (eg. "Header 1", "Project", "Task"). Instead it allows the user/application to define those categories. The ultimate goal is to democratize your to-do lists, notes, writing, etc. making them portable and platform-independent.
Aside from feedback on the specs & philosophy of Margin, which would be very appreciated, I could use some serious help with the parser (https://margin.love/parser/). I'm only a casual coder, so this problem was a difficult one for me. The parser is both incomplete and buggy, but hopefully it gets across the basics of how Margin is supposed to work.
Sorry, one more note because I'm currently trying to write a parser for this. You should specify very specifically what characters are ignored in the beginnings and ends of lines. I would strongly suggest using UTF-8 values to specify them, too, since that's nice and unambiguous. Include in that set of ignored characters exactly what whitespace characters you are choosing to ignore, even if it's just spaces and/or tabs. There are lots of whitespace characters. Lots.
One issue with ambiguity now is that you specify that "dashes" are ignored, even though (I assume) you probably meant "hyphens". Using UTF-8 would solve that.
Also, you might consider adding "+" the set of ignored characters. It's frequently used for lists. Just something to think about.
EDIT: I should probably start filing these as issues on your github
I like the indexes, though without a query language implied I'm not sure how useful they are. I can see why you wouldn't want to specify a query language in a markup spec though.
I made a query language for use in situations just like this years ago, never ended up using it much though.
Another question: if an item has multiple annotations, should the annotations be considered ordered? Or is ordering defined to be non-meaningful? I don’t know if there’s an obvious answer to this, but some applications might benefit from semantic ordering.
These probably seem like really picky questions but I’m thinking about how to write a parser for this and these are the questions that are coming up.
My first instinct is no, the order of annotations should not matter to a parser. Or, put more precisely: that would be outside the scope of the spec, and keeping track of annotation order shouldn't be required of a valid parser. (Though, of course, if an application wanted to keep these annotations ordered it should be allowed to do so.)
Though it's a great question, and I could see myself changing course if a good use case were pitched whereby the ordering of annotations mattered specifically in a way that a use-case-specific app couldn't function if those children weren't already ordered for it in the plaintext.
There's also a case to be made that, because annotations are just a special type of child items, and because item order is generally meaningful in Margin, then we might as well keep annotations ordered too for good measure.
This looks really interesting. For a “casual coder”, I think you did a good job designing the spec for a new markup language. That’s not easy.
One weird edge case I’m thinking about: what happens if you have two lines, A and B, and A is indented with a tab and B is indented with 3 spaces? Is A the parent of B?
More generally: how do you compare tabs and spaces in the context of indentation levels?
This is a great question and something I've pondered. I recently learned that mixing tabs and space in certain Python environments, such Google Cloud Functions, simply isn't allowed -- uploading such code will cause an error.
Unfortunately, that might be the cleanest solution for Margin, too: to simply disallow the intermingling of tabs and spaces as hierarchical tokens within a single parent.
In that scenario, you could use tabs in one part of the document and spaces in the other -- as long as all the direct children of one parent item follow the same rules.
What if you would just be a little bit stricter and say that each level of indentation is either 1 tab or 2 spaces? 3 spaces is then interpreted as one level of indentation of an item starting with a space.
Org-mode is awesome. But only in the context of Emacs. Markdown is less capable, but almost ubiquitous. I went from org-mode[^0] to markdown some years ago, and the only thing I am missing is org-tables.
Currently I am using a set of markdown editors: Ulysses[^1], Typora, iA Writer and a few others. That makes me interface agnostic. The most important thing is the content, the (markdown)text I am producing. I can't say that about text in org-mode. It's tied to Emacs.
(Edit:)
In Ulysses I started using an annotation system myself, that's why this project here caught my attention. I am experimenting with brackets, double brackets, colons, double colons. Still in the process of experimenting.
---
[^0]: was using org-mode to write documentation, journal entries, and as a task management system
[^1]: Ulysses is way more than an editor. It is like Evernote with markdown targeting authors. It‘s my ‚Zettelkasten‘
You can type Org mode in vim, notepad.exe, Atom, Notepad++, and all other text editors out there. And in my opinion it does have advantages compared to the other, common lightweight markup standards such as Markdown, AsciiDoc, Wikitext or reStructuredText.
Org mode looks great, thanks for calling my attention to it. For anyone looking for a brief introduction, they have a compact guide[1] on their site that took me a minute to find.
It looks great for coders who are ready to learn new syntax. Though what I'd really like to avoid with Margin is anything that makes the language more complex than it needs to be.
For example, org mode defines headlines as:
The headlines in Org start with one or more stars, on the left margin. For example:
* Top level headline
** Second level
*** Third level
With Margin, there's no concept of a "headline." For the thinker, the headline could be represented by any level of the plain text hierarchical tree. And the thinker would then be free to choose (or conform to) any Margin-based app that corresponds with their model. In other words, it works best however you want to use it.
I understand the different approach, but the ecosystem that leverages it aligns quite nicely with your goals. It’s anything from a todo app to a recipe book to a financial planner.
I doubt there’s any way to take advantage of the common objectives, but if you look at how the libraries build on the syntax, there might be lessons to be shared.
This is great, thanks for sharing. I've been looking for this to create a to-do app that can save to a text format as a backup and can seamlessly be edited through the UI or text for maximum portability, so basically a UI client for modifying this type of markup plus some added features.
Great to hear! As I said to @brigandish below (and will shamelessly repeat): if when building the app you find yourself wanting to contribute to the js parser[1] in any way (or even any non javascript-based parser), it'd be much appreciated. That's my biggest need right now, is actually writing a good parser that'll encourage people to try this out.
Btw, have you seen https://treenotation.org/ ? That is the other one I was looking, but it's more abstract and you would need to construct the markup language in it first.
No, but looking at it now, it's definitely an inspiring model for how to do something like this right.
I'd say the goals of Margin are similar, but more focused on thought that is specifically hierarchical in nature -- whereas ArchieML seems to be more focused on structuring text in key:value pairs (if I'm understanding it correctly).
The hope with Margin is that lots of people already store their notes, to do lists, and random thoughts in a format that might already be (or almost be) valid Margin. It's intentionally non-technical, and its syntax should make sense to those who don't know or care what Margin is.
wow, the syntax of ArchieML seems a lot harder to learn that something with a more regular syntax like, say, YAML. I wonder if the NYT still uses that. Also having no feedback while you write on a gdoc seems like a recipe for a disaster (maybe they had or have some sort of gdoc plugin to give syntax feedback?)
Comments
Hey Show HN!
I'd love some early feedback on Margin, a markup language for hierarchically structured thought.
Margin came out of my desire to build a lightweight to-do app – though I soon realized what I really wanted was a markup language to capture structured thought. One that was not only human-readable, but also easily machine parsable.
Most importantly, Margin doesn't impose strict hierarchical categories (eg. "Header 1", "Project", "Task"). Instead it allows the user/application to define those categories. The ultimate goal is to democratize your to-do lists, notes, writing, etc. making them portable and platform-independent.
Aside from feedback on the specs & philosophy of Margin, which would be very appreciated, I could use some serious help with the parser (https://margin.love/parser/). I'm only a casual coder, so this problem was a difficult one for me. The parser is both incomplete and buggy, but hopefully it gets across the basics of how Margin is supposed to work.
Thanks!
Sorry, one more note because I'm currently trying to write a parser for this. You should specify very specifically what characters are ignored in the beginnings and ends of lines. I would strongly suggest using UTF-8 values to specify them, too, since that's nice and unambiguous. Include in that set of ignored characters exactly what whitespace characters you are choosing to ignore, even if it's just spaces and/or tabs. There are lots of whitespace characters. Lots.
One issue with ambiguity now is that you specify that "dashes" are ignored, even though (I assume) you probably meant "hyphens". Using UTF-8 would solve that.
Also, you might consider adding "+" the set of ignored characters. It's frequently used for lists. Just something to think about.
EDIT: I should probably start filing these as issues on your github
I like the indexes, though without a query language implied I'm not sure how useful they are. I can see why you wouldn't want to specify a query language in a markup spec though.
I made a query language for use in situations just like this years ago, never ended up using it much though.
https://github.com/polm/philtre
Another question: if an item has multiple annotations, should the annotations be considered ordered? Or is ordering defined to be non-meaningful? I don’t know if there’s an obvious answer to this, but some applications might benefit from semantic ordering.
These probably seem like really picky questions but I’m thinking about how to write a parser for this and these are the questions that are coming up.
My first instinct is no, the order of annotations should not matter to a parser. Or, put more precisely: that would be outside the scope of the spec, and keeping track of annotation order shouldn't be required of a valid parser. (Though, of course, if an application wanted to keep these annotations ordered it should be allowed to do so.)
Though it's a great question, and I could see myself changing course if a good use case were pitched whereby the ordering of annotations mattered specifically in a way that a use-case-specific app couldn't function if those children weren't already ordered for it in the plaintext.
There's also a case to be made that, because annotations are just a special type of child items, and because item order is generally meaningful in Margin, then we might as well keep annotations ordered too for good measure.
This looks really interesting. For a “casual coder”, I think you did a good job designing the spec for a new markup language. That’s not easy.
One weird edge case I’m thinking about: what happens if you have two lines, A and B, and A is indented with a tab and B is indented with 3 spaces? Is A the parent of B?
More generally: how do you compare tabs and spaces in the context of indentation levels?
This is a great question and something I've pondered. I recently learned that mixing tabs and space in certain Python environments, such Google Cloud Functions, simply isn't allowed -- uploading such code will cause an error.
Unfortunately, that might be the cleanest solution for Margin, too: to simply disallow the intermingling of tabs and spaces as hierarchical tokens within a single parent.
In that scenario, you could use tabs in one part of the document and spaces in the other -- as long as all the direct children of one parent item follow the same rules.
What if you would just be a little bit stricter and say that each level of indentation is either 1 tab or 2 spaces? 3 spaces is then interpreted as one level of indentation of an item starting with a space.
Did you look at org-mode? That seems to me like the closest existing implementation to what you’re attempting.
Org-mode is awesome. But only in the context of Emacs. Markdown is less capable, but almost ubiquitous. I went from org-mode[^0] to markdown some years ago, and the only thing I am missing is org-tables.
Currently I am using a set of markdown editors: Ulysses[^1], Typora, iA Writer and a few others. That makes me interface agnostic. The most important thing is the content, the (markdown)text I am producing. I can't say that about text in org-mode. It's tied to Emacs.
(Edit:)
In Ulysses I started using an annotation system myself, that's why this project here caught my attention. I am experimenting with brackets, double brackets, colons, double colons. Still in the process of experimenting.
---
[^0]: was using org-mode to write documentation, journal entries, and as a task management system
[^1]: Ulysses is way more than an editor. It is like Evernote with markdown targeting authors. It‘s my ‚Zettelkasten‘
Really? I agree with [Karl Voit 2017 essay](https://karl-voit.at/2017/09/23/orgmode-as-markup-only/) regarding that. To quote:
Voit then provides a very compelling case for why Org-mode syntax is a well-designed [lightweight markup language](https://en.wikipedia.org/wiki/Lightweight_markup_language), and works well outside of Emacs.
Org mode looks great, thanks for calling my attention to it. For anyone looking for a brief introduction, they have a compact guide[1] on their site that took me a minute to find.
It looks great for coders who are ready to learn new syntax. Though what I'd really like to avoid with Margin is anything that makes the language more complex than it needs to be.
For example, org mode defines headlines as:
With Margin, there's no concept of a "headline." For the thinker, the headline could be represented by any level of the plain text hierarchical tree. And the thinker would then be free to choose (or conform to) any Margin-based app that corresponds with their model. In other words, it works best however you want to use it.[1] https://orgmode.org/guide/
I understand the different approach, but the ecosystem that leverages it aligns quite nicely with your goals. It’s anything from a todo app to a recipe book to a financial planner.
I doubt there’s any way to take advantage of the common objectives, but if you look at how the libraries build on the syntax, there might be lessons to be shared.
How are you going to distinguish (bulleted) lists from multi-line paragraphs without a minimum of syntax?
P.S.: Whitespace is syntax too ;-) -- signed, Guido van Rossum
I really like it. I like the annotations feature. And I like how the items are isolated.
Pretty much all you need. Would love to see it augmented with rich text. :)
Good work!
This is great, thanks for sharing. I've been looking for this to create a to-do app that can save to a text format as a backup and can seamlessly be edited through the UI or text for maximum portability, so basically a UI client for modifying this type of markup plus some added features.
Great to hear! As I said to @brigandish below (and will shamelessly repeat): if when building the app you find yourself wanting to contribute to the js parser[1] in any way (or even any non javascript-based parser), it'd be much appreciated. That's my biggest need right now, is actually writing a good parser that'll encourage people to try this out.
[1] https://github.com/gamburg/margin/blob/master/parser/js/Marg...
Btw, have you seen https://treenotation.org/ ? That is the other one I was looking, but it's more abstract and you would need to construct the markup language in it first.
I see, I'll post any feedback and eventually the project I'm doing. Thanks.
How do you deal with multiline data? For example, what if I wanted to add a multiline code snippet?
Important question. Still thinking through this: https://github.com/gamburg/margin/issues/2
Are you familiar with ArchieML?
No, but looking at it now, it's definitely an inspiring model for how to do something like this right.
I'd say the goals of Margin are similar, but more focused on thought that is specifically hierarchical in nature -- whereas ArchieML seems to be more focused on structuring text in key:value pairs (if I'm understanding it correctly).
The hope with Margin is that lots of people already store their notes, to do lists, and random thoughts in a format that might already be (or almost be) valid Margin. It's intentionally non-technical, and its syntax should make sense to those who don't know or care what Margin is.
wow, the syntax of ArchieML seems a lot harder to learn that something with a more regular syntax like, say, YAML. I wonder if the NYT still uses that. Also having no feedback while you write on a gdoc seems like a recipe for a disaster (maybe they had or have some sort of gdoc plugin to give syntax feedback?)
It’s still used in many news orgs. The writers mostly don’t touch the markup, so it works out.