logo
Tags down

shadow

Add values to dataframe based on existing unique values


By : HiMaK
Date : October 15 2020, 06:10 PM
it fixes the issue I have a dataframe DF with 2 columns: , Here's another way to do it:
code :
# Copy of your data
df = pd.DataFrame({
    "class": ["Sci", "Sci", "math", "math", "math", "eng", "eng", "eng", "Hist", "Hist"],
    "student": ["Francy", "vacant", "Alex", "Arthur", "Katy", "Jack", "vacant", "Francy", "Francy", "Francy"]
    })

# An identical DF with all students equal to "vacant"
vacant_df = pd.DataFrame({"class": df["class"], "student": "vacant"})

# Remove existing 'vacant' from original DF and concatenate with de-duplicated vacant dataframe (to avoid duplicate 'vacant' entries)
final_df = pd.concat([df.loc[df.student != "vacant", vacant_df.drop_duplicates("class")])
  class student
8  Hist  Francy
9  Hist  Francy
0   Sci  Francy
1   Sci  vacant
5   eng    Jack
6   eng  vacant
7   eng  Francy
2  math    Alex
3  math  Arthur
4  math    Katy
  class student
8  Hist  Francy
9  Hist  Francy
8  Hist  vacant
0   Sci  Francy
0   Sci  vacant
5   eng    Jack
7   eng  Francy
5   eng  vacant
2  math    Alex
3  math  Arthur
4  math    Katy
2  math  vacant


Share : facebook icon twitter icon

How to create a collection of unique values based on existing array values


By : Muhammad Al-Hawari
Date : March 29 2020, 07:55 AM
wish help you to fix your issue Below code prints an Array of fileNames. , You could do this:
code :
val fileNameSet = pdfFileArray.groupBy(_.split('.').last).keys

Add a column to a dataframe using (extracting unique values) from existing columns


By : gibb_star
Date : March 29 2020, 07:55 AM
I wish this help you I am new to R, and was not able to search answers for the specific problem I have encountered. , A tidyverse approach:
code :
library(dplyr)
library(tidyr)

      # evaluate separately for each name
d %>% group_by(Name) %>% 
    # add a column of the unique values pasted together into a string
    mutate(ID = paste(unique(c(No1, No2)), collapse = ' ')) %>% 
    # separate the string into individual columns, filling with NA and converting to numbers
    separate(ID, into = paste0('ID', 1:3), fill = 'right', convert = TRUE)

## Source: local data frame [9 x 6]
## Groups: Name [3]
## 
##     Name   No1   No2   ID1   ID2   ID3
## * <fctr> <dbl> <dbl> <int> <int> <int>
## 1    Jon     1     1     1     2     3
## 2    Jon     2     1     1     2     3
## 3    Jon     3     1     1     2     3
## 4    Kel     1     2     1     2    NA
## 5    Kel     1     2     1     2    NA
## 6    Kel     1     2     1     2    NA
## 7    Don     3     3     3    NA    NA
## 8    Don     3     3     3    NA    NA
## 9    Don     3     3     3    NA    NA
# store distinct values in No1 and No2
cols <- unique(unlist(d[,-1]))
                           # split No1 and No2 by Name,
ids <- data.frame(t(sapply(split(d[,-1], d$Name), 
                           # find unique values for each split,
                           function(x){y <- unique(unlist(x))
                                       # pad with NAs,
                                       c(y, rep(NA, length(cols) - length(y)))
                           # and return a data.frame
                           }))) 
# fix column names
names(ids) <- paste0('ID', cols)
# turn rownames into column
ids$Name <- rownames(ids)
# join two data.frames on Name columns
merge(d, ids, sort = FALSE)

##   Name No1 No2 ID1 ID2 ID3
## 1  Jon   1   1   1   2   3
## 2  Jon   2   1   1   2   3
## 3  Jon   3   1   1   2   3
## 4  Kel   1   2   1   2  NA
## 5  Kel   1   2   1   2  NA
## 6  Kel   1   2   1   2  NA
## 7  Don   3   3   3  NA  NA
## 8  Don   3   3   3  NA  NA
## 9  Don   3   3   3  NA  NA
# copy d so as not to distort original with factor columns
d_f <- d
# make No* columns factors to ensure similar table structure
d_f[, -1] <- lapply(d[,-1], factor, levels = unique(unlist(d[, -1])))
# make tables of cols, sum to aggregate occurrences, and set as boolean mask for > 0
tab <- Reduce(`+`, lapply(d_f[, -1], table, d_f$Name)) > 0
# replace all TRUE values with values they tabulated
tab <- tab * matrix(as.integer(rownames(tab)), nrow = nrow(tab), ncol = ncol(tab))
# replace 0s with NAs
tab[tab == 0] <- NA
# store column names
cols <- paste0('ID', rownames(tab))
# sort each row, keeping NAs
tab <- data.frame(t(apply(tab, 2, sort, na.last = T)))
# apply stored column names
names(tab) <- cols
# turn rownames into column
tab$Name <- rownames(tab)
# join two data.frames on Name columns
merge(d, tab, sort = FALSE)

Randomly draw rows from dataframe based on unique values and column values


By : Roman
Date : March 29 2020, 07:55 AM
will help you Perhaps there is a clever way to sample, but here's a straightforward idea to get you started in the meanwhile:
code :
setDT(df)
setkey(df, session)

usedsessions = 0 # some value that's not a session number
df[, {
       res = .SD[!.(usedsessions)][sample(.N, 2)]
       usedsessions = c(usedsessions, res$session)
       res
     }
   , by = .(trt, individual)]
#    trt individual session     data
# 1:   A        Bob       7 4.256668
# 2:   A        Bob      25 2.431821
# 3:   A      Nancy      16 4.785859
# 4:   A      Nancy      19 4.865248
# 5:   A        Tim       4 3.303689
# 6:   A        Tim      13 3.550261
# 7:   B        Bob      26 3.987136
# 8:   B        Bob      17 3.283055
# 9:   B      Nancy      14 3.177226
#10:   B      Nancy       2 3.639542
#11:   B        Tim       8 2.168447
#12:   B        Tim       5 3.521123
#13:   C        Bob      21 3.284245
#14:   C        Bob      12 5.773098
#15:   C      Nancy      24 4.624428
#16:   C      Nancy       9 3.235467
#17:   C        Tim      18 4.001395
#18:   C        Tim      27 5.002110

Create a dataframe based on another dataframe using unique values


By : Tiger
Date : March 29 2020, 07:55 AM
Hope this helps First create new column by DataFrame.assign filled by 1, then create MultiIndex.from_product by Series.unique values of both columns and after DataFrame.set_index use DataFrame.reindex - there is possible set value for new appended rows in colC column by fill_value parameter:
code :
mux = pd.MultiIndex.from_product([df['colA'].unique(),
                                  df['colB'].unique()], names=['colA','colB'])
df1 = df.assign(colC = 1).set_index(['colA','colB']).reindex(mux, fill_value=0).reset_index()
print (df1)
  colA  colB  colC
0      A  A1     1
1      A  C1     0
2      A  B1     1
3      B  A1     1
4      B  C1     1
5      B  B1     0
df1 = (df.assign(colC = 1)
         .set_index(['colA','colB'])['colC']
         .unstack(fill_value=0)
         .stack()
         .reset_index(name='ColC'))

print (df1)
  colA colB  ColC
0    A   A1     1
1    A   B1     1
2    A   C1     0
3    B   A1     1
4    B   B1     0
5    B   C1     1
from  itertools import product
df1 = pd.DataFrame(product(df['colA'].unique(), df['colB'].unique()), columns=['colA','colB'])
df = df1.merge(df, how='left', indicator=True).rename(columns={'_merge':'colC'})
df['colC'] = df['colC'].eq('both').astype(int)
print (df)
  colA colB  colC
0    A   A1     1
1    A   C1     0
2    A   B1     1
3    B   A1     1
4    B   C1     1
5    B   B1     0
df1 = df1.sort_values(['colA','colB'])

new column in pandas DataFrame based on unique values (lists) of an existing column


By : npk
Date : March 29 2020, 07:55 AM
help you fix your problem Method one from sklearn, since you already have the list type column in your dfs
code :
from sklearn.preprocessing import MultiLabelBinarizer
mlb = MultiLabelBinarizer()
yourdf=pd.DataFrame(mlb.fit_transform(df['tokens']),columns=mlb.classes_, index=df.index)
df['tokens'].explode().str.get_dummies().sum(level=0).add_prefix('tokens_')
   tokens_A  tokens_B  tokens_C  tokens_D  tokens_Z
0         1         1         1         1         0
1         1         1         0         0         0
2         0         0         0         0         1
3         0         0         0         0         0
4         0         0         0         1         1
5         0         0         0         0         0
pd.get_dummies(pd.DataFrame(df.tokens.tolist()),prefix='tokens',prefix_sep='_').sum(level=0,axis=1)
   tokens_A  tokens_D  tokens_Z  tokens_B  tokens_C
0         1         1         0         1         1
1         1         0         0         1         0
2         0         0         1         0         0
3         0         0         0         0         0
4         0         1         1         0         0
5         0         0         0         0         0
df['tokens'].explode().str.get_dummies().sum(level=0).add_prefix('tokens_')
   tokens_England  tokens_Germany  tokens_Japan  tokens_Morocco  tokens_Spain
0               1               1             1               0             1
1               0               1             0               0             1
2               0               0             0               1             0
3               0               0             0               0             0
4               1               0             1               0             0
5               0               0             0               0             0
Related Posts Related Posts :
  • Python hex string encoding
  • Get week start date from week number
  • How to use imports from requirements.txt in python
  • Removing tab indent in ipython shell
  • I need to remove duplicates from a list but add the numeric value in them
  • Delay default arguments being read until function is called
  • Interpolate / fillna with a decay formula in pandas
  • What python package can translate Greek letter to ASCII requivalent?
  • How to get output of OS command from Jupyter notebook?
  • Printing AND writing the RIGHTLY formatted number
  • How do I create a shortcut to import most used python modules?
  • Matplotlib: Show selected date labels on x axis
  • Understanding memoization in Python
  • why does the len function return 2 on some iterations when they are all the same length?
  • Change in preference value does not affect the results of Affinity propagation Clustering
  • returning values inside a function
  • Why cant I use a variable in str slicing?
  • Section divider in Spyder
  • Conditional statement in selenium if element does not exists
  • Pandas : how to select index/row label in dataframe that matches a condition
  • What does zero do in A[0] in this code? Why not empty or another number?
  • Google App Engine urlfetch PayloadTooLargeError: Request exceeds 10 MiB limit for URL
  • Is there a way to set up optional arguments to bypass input arguments?
  • Suppress OpenMP debug messages when running Tensorflow on CPU
  • How to do GridSearchCV for F1-score in classification problem with scikit-learn?
  • Why does .pop() eventually stop and not keep removing items from a list until the list is empty?
  • How do I acess my Spider data from my main.py script?
  • Python Pandas Expand a Column of List of Lists to Two New Column
  • Overhead of python multiprocessing initialization is worse than benefits
  • Python Joining List and adding and removing characters
  • Adding an lxml library to project
  • Concatenating tensors in Tensorflow with None axis
  • Need help understanding why i get attribute error
  • How to force a MIDI device to report control status?
  • What does *** mean in Python -3?
  • How to get GFCC instead of MFCC in python?
  • How do I print a number n times in python?
  • How do i split a string wherever there are digits?
  • List Comprehension Python Prime numbers
  • "list index out of range" when reading data from file
  • What's the correct datetime format for the specified date string?
  • I cannot import CSV file?
  • Matplotlib pyplot plots look different after calling pandas profiling. How can I fix this?
  • Stopping all the instances of a specific region
  • Deal with Birtish summer time
  • Unable to use ColorWheel without loading kv (AttributeError)
  • What are these characters called: 。. !?etc Trying to split sentences stops working with non standard characters
  • rand.randint returning same number over and over?
  • Find longest sequence that does not contain a certain number
  • How do I convert a map object to list and also assign to a variable
  • sympy error: 'Symbol' object has no attribute 'pi'
  • How to remove words without vowels from a list in python
  • Downloading python to macbook
  • TypeError: __init__() missing 1 required positional argument: 'units'
  • Check if a class is a dataclass in Python
  • Unable to scrape google news heading via their class
  • Array of structs with dynamic allocation runs very slow in C in comparison to Python
  • Python Pandas - find all unique combinations of rows of a DataFrame without repeating values in the columns
  • How do I change the numbers in a cell to the word 'Bus' in Pandas Python
  • 'ascii' codec can't encode character : ordinal not in range (128)
  • shadow
    Privacy Policy - Terms - Contact Us © 35dp-dentalpractice.co.uk