lysecret on What would it take to recreate dplyr in Python? (2020)

Comment on What would it take to recreate dplyr in Python? (2020)parent

I have been using indexes more and more if you do essentially series based operations but you need to re-associate the results back. It enables you to avoid using dataframes as inputs to your data transformations.

sweezyjeezy 4y

I am experienced with pandas and I understand its uses - but it definitely makes learning it much more confusing for new people - like the first time you see a multi-index you're just like 'OMG what'. It also feels like it silghtly breaks the mental model of a dataframe for me - like why am I treating these columns as special all of a sudden? Sometimes that can make things slicker, but 90% of the time for me it just necessitates the need for .reset_index or index=False or equivalent. If I want to use index to optimise something - that should be a deliberate act, not something pushed on me by the API.

cardosof 4y

As someone with years of working experience with dplyr who had to learn pandas, 100% agree. And thanks for that post, I thought it was just me.

lysecret 4y

yea I know what you mean. The main use I have found is something like. Say you have a price per company_id and per transaction_id. Now you have some function, which for the sake of it, takes it to an exponent.

i used to write a lot of functions which went like this:

  def convert_price(price_frame:pd.DataFrame) -> pd.DataFrame:
      price_frame["new_price"] = np.exp(price_frame)
      return price_frame

and i came back to these functions, and i was always like. hmm what needs to be in the price frame. which columns etc. Also it mutates the state of the data frame

while i now write functions like

  def convert_price(price:pd.Series) -> pd.Series:
      return np.exp(price)

However what is good as well you can treat the series as a single dimensional array and do operations on it. Its not a perfect example since im not using the ids haha but you might see what i mean :D

tgb 4y

I think it helps coming from another direction where dataframes are fancy 2D numpy arrays or a fancy 'dict-of-dicts'. I do like being able to query from index with .loc. Nonetheless I basically agree with you.

Comments