logo
Tags down

shadow

How can I search several elements in python list?


By : santiago
Date : October 16 2020, 06:10 PM
To fix this issue I want to write a program to find how many [a,b,c] in [a,a, b,c,d,a,a,b,c,c,d,d,d]. , Here's one approach:
code :
l = ['a','b','c','d','a','a','b','c','c','d','d','d']
seq = ['a','b','c']

sum(1 for i in range(len(l) - len(seq)-1) if l[i: i + len(seq)] == seq)
# 2
[l[i: i + len(seq)] for i in range(len(l) - len(seq)-1)]

[['a', 'b', 'c'],
 ['b', 'c', 'd'],
 ['c', 'd', 'a'],
 ['d', 'a', 'a'],
 ['a', 'a', 'b'],
 ['a', 'b', 'c'],
 ['b', 'c', 'c'],
 ['c', 'c', 'd']]
''.join(l).count(''.join(seq))
# 2


Share : facebook icon twitter icon

How to search for each elements of a list in a string in python


By : user2750413
Date : March 29 2020, 07:55 AM
I think the issue was by ths following , let's say there's a list
code :
vowels = {'a', 'e', 'i', 'o', 'u'}
x  =input('Enter...')
new_string = ''.join(c for c in x if c not in vowels)

Text search elements in a big python list


By : Sukesh K
Date : March 29 2020, 07:55 AM
should help you out Make a list comprehension:
code :
[line for line in cell_lines if "KIDNEY" in line]
{
    "KIDNEY": ["769P_KIDNEY","786O_KIDNEY"],
    "NERVOUS_SYSTEM": ["LN18_CENTRAL_NERVOUS_SYSTEM"]
}

Search a Python list for a specific string using unknown elements in list


By : Jeff
Date : March 29 2020, 07:55 AM
hop of those help? I have a list that holds multiple, duplicate strings that come from a .csv file: , You can just cast it to a set. Like this:
code :
listTwo = set(listOne)
print(listTwo)

How to efficiently search list elements in a string in python


By : user2363173
Date : March 29 2020, 07:55 AM
I hope this helps . Solution provided below has approximately O(n) complexity when it comes to runtime, where n is the number of tokens in each sentence.
For 5 million sentences and your concepts.txt it performs required operations in ~30 seconds, see basic test in the third section.
code :
concepts = [
    ["natural language processing", "text mining", "texts", "nlp"],
    ["advanced data mining", "data mining", "data"],
    ["discourse analysis", "learning analytics", "mooc"],
]
for concept in concepts:
    concept[1:] = concept[0]
["advanced data mining", "data something", "data"]
"Here is data something and another data"
"Here is advanced data mapping and another advanced data mining"
"Here is advanced data mapping something and another advanced data mining"
structure = [
    {"data": {"something": "advanced data mining"}},
    {"data": "advanced data mining"},
]
def get_longest(concepts: List[List[str]]):
    return max(len(text.split()) for concept in concepts for text in concept[1:])
def init_hierarchical_dictionaries(longest: int):
    return [(length, {}) for length in reversed(range(longest))]
def create_hierarchical_dictionaries(concepts: List[List[str]]):
    # Initialization
    longest = get_longest(concepts)
    hierarchical_dictionaries = init_hierarchical_dictionaries(longest)

    for concept in concepts:
        for text in concept[1:]:
            tokens = text.split()
            # Initialize dictionary; get the one with corresponding length.
            # The longer, the earlier it is in the hierarchy
            current_dictionary = hierarchical_dictionaries[longest - len(tokens)][1]
            # All of the tokens except the last one are another dictionary mapping to
            # the next token in concept.
            for token in tokens[:-1]:
                current_dictionary[token] = {}
                current_dictionary = current_dictionary[token]

            # Last token is mapped to the first concept
            current_dictionary[tokens[-1]] = concept[0].split()

    return hierarchical_dictionaries
def embed_sentences(sentences: List[str], hierarchical_dictionaries):
    return (traverse(sentence, hierarchical_dictionaries) for sentence in sentences)
def traverse(sentence: str, hierarchical_dictionaries):
    # Get all tokens in the sentence
    tokens = sentence.split()
    output_sentence = []
    # Initialize index to the first token
    index = 0
    # Until any tokens left to check for concepts
    while index < len(tokens):
        # Iterate over hierarchical dictionaries (elements of the array)
        for hierarchical_dictionary_tuple in hierarchical_dictionaries:
            # New index is returned based on match and token-wise length of concept
            index, concept = traverse_through_dictionary(
                index, tokens, hierarchical_dictionary_tuple
            )
            # Concept was found in current hierarchical_dictionary_tuple, let's add it
            # to output
            if concept is not None:
                output_sentence.extend(concept)
                # No need to check other hierarchical dictionaries for matching concept
                break
        # Token (and it's next tokens) do not match with any concept, return original
        else:
            output_sentence.append(tokens[index])
        # Increment index in order to move to the next token
        index += 1

    # Join list of tokens into a sentence
    return " ".join(output_sentence)
def traverse_through_dictionary(index, tokens, hierarchical_dictionary_tuple):
    # Get the level of nested dictionaries and initial dictionary
    length, current_dictionary = hierarchical_dictionary_tuple
    # inner_index will loop through tokens until match or no match was found
    inner_index = index
    for _ in range(length):
        # Get next nested dictionary and move inner_index to the next token
        current_dictionary = current_dictionary.get(tokens[inner_index])
        inner_index += 1
        # If no match was found in any level of dictionary
        # Return current index in sentence and None representing lack of concept.
        if current_dictionary is None or inner_index >= len(tokens):
            return index, None

    # If everything went fine through all nested dictionaries, check whether
    # last token corresponds to concept
    concept = current_dictionary.get(tokens[inner_index])
    if concept is None:
        return index, None
    # If so, return inner_index (we have moved length tokens, so we have to update it)
    return inner_index, concept
import ast
import time
from typing import List


def get_longest(concepts: List[List[str]]):
    return max(len(text.split()) for concept in concepts for text in concept[1:])


def init_hierarchical_dictionaries(longest: int):
    return [(length, {}) for length in reversed(range(longest))]


def create_hierarchical_dictionaries(concepts: List[List[str]]):
    # Initialization
    longest = get_longest(concepts)
    hierarchical_dictionaries = init_hierarchical_dictionaries(longest)

    for concept in concepts:
        for text in concept[1:]:
            tokens = text.split()
            # Initialize dictionary; get the one with corresponding length.
            # The longer, the earlier it is in the hierarchy
            current_dictionary = hierarchical_dictionaries[longest - len(tokens)][1]
            # All of the tokens except the last one are another dictionary mapping to
            # the next token in concept.
            for token in tokens[:-1]:
                current_dictionary[token] = {}
                current_dictionary = current_dictionary[token]

            # Last token is mapped to the first concept
            current_dictionary[tokens[-1]] = concept[0].split()

    return hierarchical_dictionaries


def traverse_through_dictionary(index, tokens, hierarchical_dictionary_tuple):
    # Get the level of nested dictionaries and initial dictionary
    length, current_dictionary = hierarchical_dictionary_tuple
    # inner_index will loop through tokens until match or no match was found
    inner_index = index
    for _ in range(length):
        # Get next nested dictionary and move inner_index to the next token
        current_dictionary = current_dictionary.get(tokens[inner_index])
        inner_index += 1
        # If no match was found in any level of dictionary
        # Return current index in sentence and None representing lack of concept.
        if current_dictionary is None or inner_index >= len(tokens):
            return index, None

    # If everything went fine through all nested dictionaries, check whether
    # last token corresponds to concept
    concept = current_dictionary.get(tokens[inner_index])
    if concept is None:
        return index, None
    # If so, return inner_index (we have moved length tokens, so we have to update it)
    return inner_index, concept


def traverse(sentence: str, hierarchical_dictionaries):
    # Get all tokens in the sentence
    tokens = sentence.split()
    output_sentence = []
    # Initialize index to the first token
    index = 0
    # Until any tokens left to check for concepts
    while index < len(tokens):
        # Iterate over hierarchical dictionaries (elements of the array)
        for hierarchical_dictionary_tuple in hierarchical_dictionaries:
            # New index is returned based on match and token-wise length of concept
            index, concept = traverse_through_dictionary(
                index, tokens, hierarchical_dictionary_tuple
            )
            # Concept was found in current hierarchical_dictionary_tuple, let's add it
            # to output
            if concept is not None:
                output_sentence.extend(concept)
                # No need to check other hierarchical dictionaries for matching concept
                break
        # Token (and it's next tokens) do not match with any concept, return original
        else:
            output_sentence.append(tokens[index])
        # Increment index in order to move to the next token
        index += 1

    # Join list of tokens into a sentence
    return " ".join(output_sentence)


def embed_sentences(sentences: List[str], hierarchical_dictionaries):
    return (traverse(sentence, hierarchical_dictionaries) for sentence in sentences)


def sanity_check():
    concepts = [
        ["natural language processing", "text mining", "texts", "nlp"],
        ["advanced data mining", "data mining", "data"],
        ["discourse analysis", "learning analytics", "mooc"],
    ]
    sentences = [
        "data mining and text mining",
        "nlp is mainly used by discourse analysis community",
        "data mining in python is fun",
        "mooc data analysis involves texts",
        "data and data mining are both very interesting",
    ]

    targets = [
        "advanced data mining and natural language processing",
        "natural language processing is mainly used by discourse analysis community",
        "advanced data mining in python is fun",
        "discourse analysis advanced data mining analysis involves natural language processing",
        "advanced data mining and advanced data mining are both very interesting",
    ]

    hierarchical_dictionaries = create_hierarchical_dictionaries(concepts)

    results = list(embed_sentences(sentences, hierarchical_dictionaries))
    if results == targets:
        print("Correct results")
    else:
        print("Incorrect results")


def speed_check():
    with open("./concepts.txt") as f:
        concepts = ast.literal_eval(f.read())

    initial_sentences = [
        "data mining and text mining",
        "nlp is mainly used by discourse analysis community",
        "data mining in python is fun",
        "mooc data analysis involves texts",
        "data and data mining are both very interesting",
    ]

    sentences = initial_sentences.copy()

    for i in range(1_000_000):
        sentences += initial_sentences

    start = time.time()
    hierarchical_dictionaries = create_hierarchical_dictionaries(concepts)
    middle = time.time()
    letters = []
    for result in embed_sentences(sentences, hierarchical_dictionaries):
        letters.append(result[0].capitalize())
    end = time.time()
    print(f"Time for hierarchical creation {(middle-start) * 1000.0} ms")
    print(f"Time for embedding {(end-middle) * 1000.0} ms")
    print(f"Overall time elapsed {(end-start) * 1000.0} ms")


def main():
    sanity_check()
    speed_check()


if __name__ == "__main__":
    main()
Time for hierarchical creation 107.71822929382324 ms
Time for embedding 30460.427284240723 ms
Overall time elapsed 30568.145513534546 ms

Python: How to search an xml file for a list of elements then print them


By : Tori Morris
Date : March 29 2020, 07:55 AM
With these it helps You can iterate over children of a tag:
Related Posts Related Posts :
  • Python hex string encoding
  • Get week start date from week number
  • How to use imports from requirements.txt in python
  • Removing tab indent in ipython shell
  • I need to remove duplicates from a list but add the numeric value in them
  • Delay default arguments being read until function is called
  • Interpolate / fillna with a decay formula in pandas
  • What python package can translate Greek letter to ASCII requivalent?
  • How to get output of OS command from Jupyter notebook?
  • Printing AND writing the RIGHTLY formatted number
  • How do I create a shortcut to import most used python modules?
  • Matplotlib: Show selected date labels on x axis
  • Understanding memoization in Python
  • why does the len function return 2 on some iterations when they are all the same length?
  • Change in preference value does not affect the results of Affinity propagation Clustering
  • returning values inside a function
  • Why cant I use a variable in str slicing?
  • Section divider in Spyder
  • Conditional statement in selenium if element does not exists
  • Pandas : how to select index/row label in dataframe that matches a condition
  • What does zero do in A[0] in this code? Why not empty or another number?
  • Google App Engine urlfetch PayloadTooLargeError: Request exceeds 10 MiB limit for URL
  • Is there a way to set up optional arguments to bypass input arguments?
  • Suppress OpenMP debug messages when running Tensorflow on CPU
  • How to do GridSearchCV for F1-score in classification problem with scikit-learn?
  • Why does .pop() eventually stop and not keep removing items from a list until the list is empty?
  • How do I acess my Spider data from my main.py script?
  • Python Pandas Expand a Column of List of Lists to Two New Column
  • Overhead of python multiprocessing initialization is worse than benefits
  • Python Joining List and adding and removing characters
  • Adding an lxml library to project
  • Concatenating tensors in Tensorflow with None axis
  • Need help understanding why i get attribute error
  • How to force a MIDI device to report control status?
  • What does *** mean in Python -3?
  • How to get GFCC instead of MFCC in python?
  • How do I print a number n times in python?
  • How do i split a string wherever there are digits?
  • List Comprehension Python Prime numbers
  • "list index out of range" when reading data from file
  • What's the correct datetime format for the specified date string?
  • I cannot import CSV file?
  • Matplotlib pyplot plots look different after calling pandas profiling. How can I fix this?
  • Stopping all the instances of a specific region
  • Deal with Birtish summer time
  • Unable to use ColorWheel without loading kv (AttributeError)
  • What are these characters called: 。. !?etc Trying to split sentences stops working with non standard characters
  • rand.randint returning same number over and over?
  • Find longest sequence that does not contain a certain number
  • How do I convert a map object to list and also assign to a variable
  • sympy error: 'Symbol' object has no attribute 'pi'
  • How to remove words without vowels from a list in python
  • Downloading python to macbook
  • TypeError: __init__() missing 1 required positional argument: 'units'
  • Check if a class is a dataclass in Python
  • Unable to scrape google news heading via their class
  • Array of structs with dynamic allocation runs very slow in C in comparison to Python
  • Python Pandas - find all unique combinations of rows of a DataFrame without repeating values in the columns
  • How do I change the numbers in a cell to the word 'Bus' in Pandas Python
  • 'ascii' codec can't encode character : ordinal not in range (128)
  • shadow
    Privacy Policy - Terms - Contact Us © 35dp-dentalpractice.co.uk