pastage on Building a full-text search engine in 150 lines of Python code (2021)

Comment on Building a full-text search engine in 150 lines of Python code (2021)parent

Yeah, but what is worst? I tried to understand the Haskell way in an article last week[0]

  pure (n, guard (factor /= n) $> factor)

Which I think is more or less the same as this python line.

  return [factor for factor in factors if factor and not factor == n]

The article does fancy stuff with memory caches which I believe is easy to do in python but I need to understand the Haskell code better.

[0] Haskell: A Great Procedural Language https://entropicthoughts.com/haskell-procedural-programming#... https://news.ycombinator.com/item?id=42754098

mrkeen 1y

pure (n, guard (factor /= n) $> factor)

Returns a tuple: Left side is n, right side is (Just factor) if factor is not n, or Nothing if it is.

itishappy 1y

You're being way to clever! The machinery from the article is only needed to deal with IO.

A more direct translation:

    # py
    [token for token in tokens if token]

    -- hs
    filter (not . null) tokens

The full functions:

    # py
    def analyze(text):
      tokens = tokenize(text)
      tokens = lowercase_filter(tokens)
      tokens = stopword_filter(tokens)
      tokens = stem_filter(tokens)
      return [token for token in tokens if token]

    -- hs
    analyze :: String -> [String]
    analyze = filter (not . null) . stem_filter . stopword_filter . lowercase_filter . tokenize

pastage 1y

Thanks I will look at that too. Clever and ignorant are synonyms in my book, and I am ignorant here. In this case IO and memory was what was interesting in the article.

itishappy 1y

I'm being way to clever... I read your article when it was originally posted, but hadn't realized that you're quoting from it! My bad...

If you'll humor me, I'd love to take another shot, but from the more interesting direction!

Adding a type might clear stuff up a bit:

    -- hs
    pure (n, guard (factor /= n) $> factor) :: IO (Int, Maybe Int)

    # py
    print((n, if factor == n then factor else None))

So without the for loop, there's no list to comprehend. Adding the list back gives something more equivalent to a standard for loop in Python:

    -- hs
    for numbers $ \n ->
        pure (n, guard (factor /= n) $> factor) :: IO [(Int, Maybe Int)]

    # py
    for n in numbers:
        print((n, if factor == n then factor else None))

Which could also be written as a list comprehension in either language:

    -- hs
    [pure (n, guard (factor /= n) $> factor) | n <- numbers] :: IO [(Int, Maybe Int)]

    # py
    [print((n, if factor == n then factor else None)) for n in numbers]

Note that they behave a bit differently though, and it's the reason I haven't included `return` in the python line (what does `print` return?). Python will loop through the list and actually run the `print` function on each element, while Haskell will loop through and collect all the `IO` into one function to run later. Although it's starting to get pretty un-Pythonic, you can hack the behavior into Python with something like:

    # py
    return lambda: [f for f in (print((n, if factor == n then factor else None)) for n in numbers)]

Which would need to be run using something unholy like:

    # py
    factorize(numbers)()

Comments