logo
Tags down

shadow

Merge rows with same date to filll blanks/Nans using pandas


By : kdamani
Date : September 15 2020, 10:00 AM
wish of those help If possible more non missing values per combinations ID and Date then solution is more complicated:
code :
#because sample data     
df1 = df1.mask(df1 == 'nan')

df1 = (df1.sort_values(['ID','Date'])
          .groupby(['ID','Date'])
          .apply(lambda x: x.ffill().bfill())
          .drop_duplicates())
print (df1)
       ID       Date   Na    SO4 Mg
0  Site 1 2029-12-01    3  0.001  3
2  Site 1 2029-12-02    7  0.001  3
5  Site 1 2029-12-03    5  0.005  3
4  Site 2 2029-12-01    3  0.001  3
6  Site 2 2029-12-02  NaN  0.001  3


Share : facebook icon twitter icon

merge 2 columns in pandas dataframe filling the NaNs with the previous value


By : Harish
Date : March 29 2020, 07:55 AM
it helps some times Need ffill:
code :
df['State'] = df['State'].ffill()
print (df)
      State    RegionName
0   Alabama       Alabama
1   Alabama        Auburn
2   Alabama      Florence
3   Alabama  Jacksonville
4   Alabama    Livingston
5   Alabama    Montevallo
6   Alabama          Troy
7   Alabama    Tuscaloosa
8   Alabama      Tuskegee
9    Alaska        Alaska
10   Alaska     Fairbanks
11  Arizona       Arizona
12  Arizona     Flagstaff
13  Arizona         Tempe
14  Arizona        Tucson

Merge adjacent columns containing NaNs in Pandas


By : garlic00000
Date : March 29 2020, 07:55 AM
wish help you to fix your issue combine_first and fillna are good alternatives in general, but these alternatives work since your NaNs are exclusive.
Option 1
code :
s = quad_data.max(1)
print(s)
0     0.997652
1     0.861592
2     0.000000
3     0.997652
4     0.861592
5     2.673742
6     2.618845
7     0.432525
8          NaN
9     0.582576
10    0.508450
11    0.341510
12    0.351510
13    1.404787
14    2.410116
15    0.540265
16    1.404787
17    2.410116
18    0.540265
19    1.403903
20    1.448987
dtype: float64
s = quad_data.sum(1)
print(s)
0     0.997652
1     0.861592
2     0.000000
3     0.997652
4     0.861592
5     2.673742
6     2.618845
7     0.432525
8          NaN
9     0.582576
10    0.508450
11    0.341510
12    0.351510
13    1.404787
14    2.410116
15    0.540265
16    1.404787
17    2.410116
18    0.540265
19    1.403903
20    1.448987
dtype: float64
quad_data['new'] = s 

Merge rows with same date and add counter column in pandas


By : user1739132
Date : March 29 2020, 07:55 AM
I wish this helpful for you I have a simple DataFrame that looks like this: , Use:
code :
df.groupby(df['date'].dt.date).size().rename(columns={'size':'sum'})
df['date'] = pd.to_datetime(df['date'])

Merge rows by ID and date - Pandas


By : user3161346
Date : March 29 2020, 07:55 AM
wish help you to fix your issue I've got data frame with IDs, dates per ID and times per date. Also, a binary variable that gets 1 when weekend, 0 otherwise, and another binary variable - Y (no matter what it indicates). I want to merge rows by ID and date, and keep on weekend value per (ID, date), and get 1 on Y variable if Y got 1 in any row at (ID, date) level, 0 otherwise. , Check with max
code :
df.max(level=[0,1])

pandas merge rows based on same date


By : Kevin
Date : March 29 2020, 07:55 AM
wish help you to fix your issue You can create a new column that tracks the day of the login and then you can use groupby() and agg() to compile the values you want:
code :
df['Day'] = df['LoginTime'].str.extract(r'^(\d{4}-\d{2}-\d{2})')

df.groupby('Day').agg({'Id': 'first', 'LoginTime': 'first', 'LoginReading': 'first', 'LogoutTime': 'last',
'LogoutReading': 'last', 'Available': 'first', 'Calculated': 'sum'}).reset_index(drop=True)
      Id                    LoginTime  LoginReading  \
0  10036  2019-11-06 10:37:18.3743184       5054.68   
1  10036  2019-11-11 12:46:20.7018683       4797.39   
2  10036  2019-11-14 08:08:00.6290260       5080.59   
3  10036  2019-11-15 06:00:48.4777280       5185.65   

                    LogoutTime  LogoutReading  Available  Calculated  
0  2019-11-06 14:11:52.4833904        5057.94        500         530  
1  2019-11-11 18:09:55.8326356        4892.33        500        1772  
2  2019-11-14 18:43:31.8609822        5185.65        500        2528  
3  2019-11-15 06:31:55.0281168        5199.28        500         260  
Related Posts Related Posts :
  • Unicodedata.normalize : TypeError: normalize() argument 2 must be str, not list
  • What uses the memory of my python process? (RSS vs VMS)
  • Pandas: filter dataframe by multiple conditions with column containing nan
  • Selenium webdriver.Remote driver does not work with tor proxy(webdriver.Chrome does)
  • generate keyword arguments from positional arguments in python
  • Find all words including those with special characters
  • The total maximum value of the value chosen
  • Nested while Loops is not working in python
  • Why do my python sklearn logistic regression results differ from this example?
  • Python Regex: How do I use regular expression to read in a file with multiple lines, and extract words from each line to
  • What is the use of drop_first in pandas?
  • Is it possible to pass a Flask session to another .py-File which is not in the routing?
  • tensorflow_hub to pull BERT embedding on windows machine - extending to albert
  • Python Pandas slicing with various datatypes
  • Pandas: Checking and changing all items in a column
  • Why does __call__ returned values get garbage collected when calling a class twice: SomeClass()()
  • Insert element at every nth location in list of lists
  • PD Read in Jupyter Notebook 3.7
  • Visualize Results of each iteration of While Loop into a Time Series Chart
  • Run a function for each row and create a new Column Pandas Dataframe
  • How can I create a small IDLE-like Python Shell in Tkinter?
  • extract variable and data from a string in python
  • CUDA implementation of Softmax
  • The function to_excel of pandas generate an unexpected TypeError
  • string is contain with newline symbol (\n), how to use regex to replace \n to \n?
  • How can I use %s to replace text within a file in python?
  • How to Reference a Pandas Column that has a dot in the name
  • How to use tuple as a key of a dictionary
  • How to extract two integer values from a column of a dataframe
  • How properly build a class in the __new__ with type(3 args) and 2 ancestors?
  • How to declare the return of a function as the default parameter to another function without calling the first function?
  • Elegant way to check arguments across multiple functions
  • How can I replace elemts of a list with other elements
  • i want to use variable globally in veiws.py
  • Pandas data not being plotted
  • Python Generator: How do I generate pairs from two different lists based on user input (of how many pairs to print)
  • Python: How to use a dictionary to call methods (values in dictionary) to run based on user input (key in dictionary) in
  • Read lines between two keywords Python
  • How do you insert data from the user into the file with the most optimal using Python?
  • How do you create a loop that will work in Snowflake?
  • Why can't I change the __class__ attribute of an instance of object?
  • Concatenating pandas dataframes from pickle vs. from in-memory dictionary - why does in-memory fail?
  • How to Calculate time difference between two date columns
  • In '<string>' requires string as left operand, not list
  • Django clean() change field requirement
  • Python - TypeError: write() argument must be str, not bytes
  • Commutative Count in a groupby dataframe on other columns condition
  • Undo np.fft.fft2 to get the original image
  • What is the proper way to share a program without sharing personal information?
  • Pandas DataFrame - summing rows by multiple column values
  • Python - best approach to mapping codes in data to description
  • I need to know how to do this, but it may be impossible
  • pandas dataframe columns with list values
  • Wrong value of standard deviation
  • Django POST error: tuple has no attribute get, despite similar code working previously
  • Convert a 3d-array into 2d-array
  • how to find the shortest distance between two regex
  • Python unit testing on class methods with no input arguments
  • Bokeh: Repeated plotting in jupyter lab increases (browser) memory usage
  • Cannot convert string into float in a for loop
  • shadow
    Privacy Policy - Terms - Contact Us © 35dp-dentalpractice.co.uk