Skip to content

Comment on Ask HN: Who is using Common Lisp?parent

Comments

Just a few deficiencies that I found:

1. Jisho can't really do conjugations. It only says "X could possibly be an inflection of Y with these forms" based on the ending of the verb. For example 邪推してしまう becomes

邪推してしまう could be an inflection of 邪推する, with these forms: Te-form. It is a connective form of the verb, usually called the gerund.

As you can see it only saw "邪推して" and ignored all the rest. ichi.moe always fully parses each conjugation.

2. Jisho can't handle when words are spelled in kana, unless the word is specifically marked as "usually kana". For example when parsing a sentence "ヤバそうなはなしを聞いた", the word "ヤバそう" is not recognized at all, and "なはなし" is parsed as "なは"+"なし". ichi.moe can often handle such spellings.

3. It splits expressions like "いざとなったら" in "いざとなったら勇気が無くなった" (despite the word obviously being present in their dictionary) and the meaning of the expression can be hard to guess from the components.

Considering their segmenter apparently last updated in 2013 there's little hope this will ever improve.

When I do echo '邪推してしまう' | mecab I get

  邪推	名詞,サ変名詞,*,*,邪推,じゃすい,代表表記:邪推/じゃすい カテゴリ:抽象物
  して	動詞,*,サ変動詞,タ系連用テ形,する,して,代表表記:する/する 付属動詞候補(基本) 自他動詞:自:成る/なる
  しまう	接尾辞,動詞性接尾辞,子音動詞ワ行,基本形,しまう,しまう,連語
  EOS
and I have no idea why jisho.org completely ignores the しまう.

The other problems do seem to be related to the segmentation approach MeCab takes. I've usually found it to be satisfactory, but maybe I just haven't been looking hard enough for errors. I might try and see how well your Ichiran works as a replacement. Thanks!

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.