Repost of my comment on the blog (because waiting in the queue and not approved).
You should take the time to discuss a bit with your librarian. As I did my PhD in Denmark (DTU), I naively wrote a robot to download the issues of a well known chemical data journal. In about a week of balanced usage, I went to discuss with our librarian. He had seen my usage, was nice not to talk about it, but told me this: I downloaded more than the entire university in a year… and it was not a lot. It means that at that time, they paid a bit less than the $35 per article price.
What is really important to notice is that Elsevier are not selling knowledge for most of the scientific communities but influence. That is, you are published, cited, you get ranking and your university reward you. This is what we need to address if we want to have really open access. We need a better way to “sell” influence to the university researchers and deans.
As I am building Cheméo http://chemeo.com a chemical data search engine, I suffer too. It is maybe time to unit and propose a legal, efficient and rewarding way for the researchers to publish their papers. We can do that on the side and let our influence grow.
Additional notes for HN readers as yeah, we are a bit more on the programming side. What we basically need is a parallel DOI system easy to use, able to load all the open repositories and able to accept "direct" submissions.
We are not going to solve the problem in a year, this is an influence issue, it will take time, years, to really address it, be it by our own work or by "law".
How come your search system doesn't appear to know chemistry? That is, I searched for "CCO" and found ethyl alcohol, but I searched for "OCC" and found nothing. Are you only doing a text search on the SMILES rather than a canonicalization first?
Good point, this is on the way. The problem is more to do it right, that is, you want for each "word" to detect if this is a word, a chemical formula or a SMILES. Then, if SMILES, you then need to canonicalize it and search.
Nothing very complicated, but it needs to be well done to be of any use.
Plus, to detect if its a systemic, or semi-systemic name, and extract the structure from that. I know of three tools which do that, and only one is free.
If you want a SMILES detector, you can use my opensmiles-ragel grammar to detect if a word is syntactically correct. (Not grammatically correct; it will allow "c1C" unless you write code to require balanced parentheses and matching ring counts.)
This would be much faster than passing it to any of the cheminformatics toolkits to do the first level detection.
If you're doing similarity searches, you might be interested in my chemfp project.
Comments
Repost of my comment on the blog (because waiting in the queue and not approved).
You should take the time to discuss a bit with your librarian. As I did my PhD in Denmark (DTU), I naively wrote a robot to download the issues of a well known chemical data journal. In about a week of balanced usage, I went to discuss with our librarian. He had seen my usage, was nice not to talk about it, but told me this: I downloaded more than the entire university in a year… and it was not a lot. It means that at that time, they paid a bit less than the $35 per article price.
What is really important to notice is that Elsevier are not selling knowledge for most of the scientific communities but influence. That is, you are published, cited, you get ranking and your university reward you. This is what we need to address if we want to have really open access. We need a better way to “sell” influence to the university researchers and deans.
As I am building Cheméo http://chemeo.com a chemical data search engine, I suffer too. It is maybe time to unit and propose a legal, efficient and rewarding way for the researchers to publish their papers. We can do that on the side and let our influence grow.
Additional notes for HN readers as yeah, we are a bit more on the programming side. What we basically need is a parallel DOI system easy to use, able to load all the open repositories and able to accept "direct" submissions.
We are not going to solve the problem in a year, this is an influence issue, it will take time, years, to really address it, be it by our own work or by "law".
How come your search system doesn't appear to know chemistry? That is, I searched for "CCO" and found ethyl alcohol, but I searched for "OCC" and found nothing. Are you only doing a text search on the SMILES rather than a canonicalization first?
Good point, this is on the way. The problem is more to do it right, that is, you want for each "word" to detect if this is a word, a chemical formula or a SMILES. Then, if SMILES, you then need to canonicalize it and search.
Nothing very complicated, but it needs to be well done to be of any use.
Thanks for the feedback!
Plus, to detect if its a systemic, or semi-systemic name, and extract the structure from that. I know of three tools which do that, and only one is free.
If you want a SMILES detector, you can use my opensmiles-ragel grammar to detect if a word is syntactically correct. (Not grammatically correct; it will allow "c1C" unless you write code to require balanced parentheses and matching ring counts.)
This would be much faster than passing it to any of the cheminformatics toolkits to do the first level detection.
If you're doing similarity searches, you might be interested in my chemfp project.