[python] cannot drop rows in dataFrame

Started by
3 comments, last by suliman 4 years ago

Hi!

I have two problems with my code that used to work fine, maybe python updated or something. I have a dataFrame called data and try to drop rows that has ‘plant name’ either as ‘0’ or is empty. This is the code that fails now:

    data = df.iloc[start_row-1:end_row-1, start_col-1:end_col]
    names = df.iloc[start_row-2:start_row-1, start_col-1:end_col].astype(str)
    names = names.values.tolist()
    data.columns = names
    data = data.reset_index(drop=True)
    
    data = data[data['plant name'].notnull()]
    data = data[data['plant name'] != '0']

It gives me this error for any of the last two rows

Exception: cannot handle a non-unique multi-index

I dont need it to be multi-index and im not even sure how that works. It seems it cannot handle any column having the same name (several of the column names are empty). So i fill these empty column names with index numbers (in the excel file this data comes from) and now it doesnt gove me exception anymore. BUT it still doesnt drop the empty/'0' rows (it used to do that).

  1. how can i make it except columns with empty names (im just interested in the plant name column anyway). Can I use column index instead of using the ‘plant name' identifier? (it's the forst column that has the names i check for)
  2. why doesnt it drop the rows anymore? (when i fix it so it doesnt give exception)
  3. can some other method work for me? I tried looping it, like seen below, but that does nothing (but it doesnt crash? ). The empty rows seem to be ‘nan
    in the API so maybe that is not the same as being empty? Also checking for lines where the name is ‘0’, I dont know how to do here…
    for index, row in data.iterrows():
        if row['plant name'].empty:
            data.drop(index, inplace=True)

Thanks a bunch!

Advertisement

The programming language may be Python, but whatever dataFrame is, it's non-standard Python. As such, you may be able to find better help in a forum where they know and discuss the library where ‘dataFrame’ is part of.

could you show us the data you are working on, and whether there is any preliminary code to the snippet you posted. If it is possible, try to simplify your code focusing on the problem you are facing.

If someone has a similar issue:
The problem was that when casting “names” into a list and posting it back to the columns of “data”, that made “data” multi-indexed. I dont know why but it does.

Manipulating the names without making them a list solved this for me (when I keep the dataFrame “data” single-indexed, the dropping of the rows works again.

This topic is closed to new replies.

Advertisement