logo
Tags down

shadow

Run a perl script with Python on multiple files at once in a folder


By : Hetal Trivedi
Date : October 18 2020, 06:10 PM
will help you That Perl script processes one file. Also, that string passed to shell via os.system doesn't get expanded into a valid command with a file list as intended with the * shell glob.
Instead, build the file list in Python, using os.listdir or glob.glob or os.walk. Then iterate over the list and call that Perl script on each file, if it must process only one file at a time -- or, modify the Perl script to process multiple files and run it once with the whole list.
code :
import os

data_path   = "/home/user/Desktop/data/"
output_path = "/home/user/Desktop/result/"

for file in os.listdir(data_path):
    if not file.endswith(".txt"):
        continue

    print("Processing " + file)                      # better use subprocess
    run_perlscript = "perl " + " perlscript.pl " + \
        data_path + file  + " >> " + output_path + "output.txt"
    os.system(run_perlscript)
import subprocess

with open(output_path + "output.txt", "a") as fout:
    for file in os.listdir(path):
        if not file.endswith(".txt"):
            continue 
        subprocess.run(["perl", "script.pl", data_path + file], stdout=fout)
use strict;
use warnings;
use feature 'say';    
use open ':std', ':encoding(UTF-8)';

foreach my $filename (@ARGV) {
    say "Processing $filename";

    my %count;

    open my $fh, '<', $filename  or do {
       warn "Can't open '$filename': $!";
       next;
    };
    while (<$fh>) {   
        $count{ lc $1 }++ while /(\w+)/g;
    }   
    close $fh;

    my $prn_cnt = 0;
    foreach my $word ( sort { $count{$b} <=> $count{$a} } keys %count) {   
        print "$count{$word} $word\n" if $prn_cnt++ < 10; 
    }   
}
import subprocess

data_path   = "/home/user/Desktop/data/"
output_path = "/home/user/Desktop/result/"

files = glob.glob(data_path + "*.txt")

with open(output_path + "output.txt", "a") as fout:
    subprocess.run(["perl", "script.pl", files], stdout=fout)


Share : facebook icon twitter icon

Monitor folder for new files using unix ksh shell script or perl script and trigger perl script


By : JWesker
Date : March 29 2020, 07:55 AM
it should still fix some issue Check incron. It seems to do exactly what you need.

Perl Script: Search for string from a folder with multiple files


By : Muhit Bin Elahi Mith
Date : March 29 2020, 07:55 AM
I think the issue was by ths following , I have a Folder with multiple .txt files in it. I want to check few strings in those text files and give output as a out.txt with 5 lines above and 5 lines below of the located string. , if you insist on a perl oneliner;
code :
perl -n -e 'if (/searchStringHere/) {print "\n\n\n\n\n$_\n\n\n\n\n"}' *.txt
$flag, $l1, $l2, $l3, $l4, $l5, $l6;
$cnt = 5;
$file = shift;
open("$file") || die "can't open $file\n";
while (<>) {
 $l1 = $l2; # starting with reserving the lines for back print
 $l2 = $l3;
 $l4 = $l5;
 $l5 = $l6;
 $l6 = $_;
 if (/your string here/) {
   $cnt = 5;
   print ($l1$l2$l3$l4$l5$l6);# print line + 5 prev lines
   next
 }
 if ($cnt >0) { # print the next 5 lines
    print $l6;
    $cnt--;
 } 
}

Run Python script over multiple files in a folder


By : David Burne-jones
Date : March 29 2020, 07:55 AM
To fix this issue indent the code that does the dataframe transformation and include it in the for loop like this:
code :
import glob
import pandas as pd
import numpy as np

files = glob.glob('folder/*.csv')
for file in files:
    df = pd.read_csv(file)

    #make conversion
    df['Time taken'] = pd.to_datetime(df['Time taken'])
    df['Time taken'] = df['Time taken'].dt.hour + df['Time taken'].dt.minute / 60

    #output file
    df.to_csv('updated_{}'.format(file), index = False)

How to Perl script to move files with specific extension from one folder to another and rename the moved files


By : AVS
Date : March 29 2020, 07:55 AM
Does that help I have 2 folders DIR1 DIR2 ,i need to move only files with extension ".csv" only from dir1 to dir2

Running Python script over multiple files in a folder


By : James Holder
Date : March 29 2020, 07:55 AM
it fixes the issue I use python in opensuse, my problem is i need to execute large data in my folder. for example , use bash like this:
Related Posts Related Posts :
  • RegEx for matching specific element of HTML
  • How to initiate widgets through tk/tcl
  • urlparse does not raise exception for an invalid url
  • plot stacked percentage barchart matplotlib
  • How to have the .isupper() and .islower() methods in one line of code?
  • Removing header index from dataframe
  • how to input all data first, then give all output in python?
  • Hot to fix Tensorflow model not running in Eager mode with .fit()?
  • Proxy configuration in Scrapy
  • If/else statement within loop over dataframe
  • I have a code or stop the loop, I do not know how I can do for what stops
  • python pandas : lambda or other method to count NaN values / len(value)<1 along rows
  • Combine two dataframes with same values in several columns
  • Replace Iterations by elegant Pandas code
  • If all elements match requirement not using "if all"
  • Access to 3D array in fragment shader
  • How to normalize the columns of a DataFrame using sklearn.preprocessing.normalize?
  • Validation loss not moving with MLP in Regression
  • ML with imbalanced binary dataset
  • Is there a way to iterate through s3 object content using a SQL expression?
  • Appending lists to a result list keeps replacing the last one appended
  • How can I reuse a function to order different attributes of an object
  • Finding an integer in a list of integers if condition fulfilled
  • Python: Replacing character in for loop
  • Why can I run this command from the terminal but I get an error when my python runs it from the terminal for me?
  • How to create a conda environment from global python environment?
  • PyGame MOUSEBUTTONDOWN event not registering?
  • Pandas - Replace values in column with other values from the same column
  • Why this statement is evaluated to False even it is true?
  • Return an element based on another element in a nested list in python
  • Error ::: ValueError: could not convert string to float: '28,37'
  • How to use for and if together in Python
  • How to call a python method from robot framework
  • Python ThreadPoolExecutor Suppress Exceptions
  • how can solve this problem with dynamic programming?
  • How to convert "tensor" to "numpy" array in tensorflow?
  • Tf 2.0 : RuntimeError: GradientTape.gradient can only be called once on non-persistent tapes
  • Scale and concatenate pandas dataframe into a dask dataframe
  • How to create a URL for templateView?
  • Python : Not getting simple adding result
  • Python hex string encoding
  • Get week start date from week number
  • How to use imports from requirements.txt in python
  • Removing tab indent in ipython shell
  • I need to remove duplicates from a list but add the numeric value in them
  • Delay default arguments being read until function is called
  • Interpolate / fillna with a decay formula in pandas
  • What python package can translate Greek letter to ASCII requivalent?
  • How to get output of OS command from Jupyter notebook?
  • Printing AND writing the RIGHTLY formatted number
  • How do I create a shortcut to import most used python modules?
  • Matplotlib: Show selected date labels on x axis
  • Understanding memoization in Python
  • why does the len function return 2 on some iterations when they are all the same length?
  • Change in preference value does not affect the results of Affinity propagation Clustering
  • returning values inside a function
  • Why cant I use a variable in str slicing?
  • Section divider in Spyder
  • Conditional statement in selenium if element does not exists
  • Pandas : how to select index/row label in dataframe that matches a condition
  • shadow
    Privacy Policy - Terms - Contact Us © 35dp-dentalpractice.co.uk