logo
Tags down

shadow

Connect Cloud Storage to Cloud Function


By : R.Sinan
Date : September 16 2020, 08:00 PM
like below fixes the issue My advice is to not use JSON key file in your Cloud Functions (and on all GCP product). With Cloud Function, like with other GCP product, you have the capability to load automatically the service account during your deployment.
The advantage of Cloud Function Identity is that you haven't a key file to store secretly, you don't have to rotate your key file for increasing the security, you don't have risk of leak of key file,...
code :
import google.auth

credentials, project_id = google.auth.default()


Share : facebook icon twitter icon

How do I read the contents of a new cloud storage file of type .json from within a cloud function?


By : Krunal Panchbhaya
Date : March 29 2020, 07:55 AM
I hope this helps you . You need to use a client library for google storage instead of accessing via the URL. Using request() against the URL would only work if the file was exposed to public access.
Import the google cloud storage library in the npm-managed directory containing your project.
code :
npm i @google-cloud/storage -S
const storage = require('@google-cloud/storage')();
const storage = require('@google-cloud/storage')();

exports.logNewJSONFiles = function logNewJSONFiles(event){
    return new Promise(function(resolve, reject){
        const file = event.data;
        if (!file){
            console.log("not a file event");
            return resolve();
        }
        if (file.resourceState === 'not_exists'){
            console.log("file deletion event");
            return resolve();
        }
        if (file.contentType !== 'application/json'){
            console.log("not a json file");
            return resolve();
        }
        if (!file.bucket){
            console.log("bucket not provided");
            return resolve();
        }
        if (!file.name){
            console.log("file name not provided");
            return resolve();
        }
        (storage
         .bucket(file.bucket)
         .file(file.name)
         .download()
         .then(function(data){
             if (data)
                 return data.toString('utf-8');
         })
         .then(function(data){
             if (data) {
                 console.log("new file "+file.name);
                 console.log(data);
                 resolve(data);
             }
         })
         .catch(function(e){ reject(e); })
             );
    });
};
gcloud beta functions deploy logNewJSONFiles --stage-bucket gs://my-stage-bucket --trigger-bucket gs://my-echo-bucket

How to write to a cloud storage bucket with a firebase cloud function triggered from firestore?


By : user1487776
Date : March 29 2020, 07:55 AM
will help you The issue was that I mistakenly thought the call to bucket was to set the subdirectory inside the bucket. Instead of bucket('signatures') I should have made it an empty call like bucket() and then provided an options parameter (like { destination: '/signatures/temp.txt' }) for the upload call:
code :
const functions = require('firebase-functions');
const os = require('os');
const fs = require('fs');
const path = require('path');
const admin = require('firebase-admin');
admin.initializeApp();

exports.testItOut = functions.firestore
    .document('blarg/{docId}')
    .onUpdate((change, context) => {
    console.log( "Inside #testItOut" );
    const storage = admin.storage()
    let fileName = 'temp.txt';
    let destination = '/signatures/temp.txt';
    const tempFilePath = path.join(os.tmpdir(), fileName);
    console.log( `Writing out to ${tempFilePath}` );
    fs.writeFileSync(tempFilePath, "something!" );

    return storage
        .bucket()
        .upload( tempFilePath, { destination } )
        .then( () => fs.unlinkSync(tempFilePath) )
        .catch(err => console.error('ERROR inside upload: ', err) );
    });

How can I use a Google Cloud Function to push a file from a Cloud Storage bucket into an instance?


By : user2319167
Date : March 29 2020, 07:55 AM
help you fix your problem You can trigger a Cloud Function from a Google Cloud Storage bucket, and by selecting the Event Type to be Finalize/Create, each time a file is uploaded in the bucket, the Cloud Function will be called.
Each time a new object is created in the bucket, the cloud function will receive a notification with a Cloud Storage object format.
code :
import requests
from google.cloud import storage

def hello_gcs(data, context):
    """Background Cloud Function to be triggered by Cloud Storage.  
    Args:
        data (dict): The Cloud Functions event payload.
        context (google.cloud.functions.Context): Metadata of triggering event.
    Returns:
        None; the file is sent as a request to 
    """
    print('Bucket: {}'.format(data['bucket']))
    print('File: {}'.format(data['name']))

    client = storage.Client()
    bucket = client.get_bucket(format(data['bucket']))
    blob = bucket.get_blob(format(data['name']))

    contents = blob.download_as_string()

    headers = {
        'Content-type': 'text/plain',
    }

    data = '{"text":"{}"}'.format(contents)
    response = requests.post('https://your-instance-server/endpoint-to-download-files', headers=headers, data=data)
    return "Request sent to your instance with the data of the object"
google-cloud-storage
requests
sudo apt-get update
sudo apt-get install apache2
sudo apt-get install libapache2-mod-wsgi
sudo apt-get install python-pip
sudo pip install flask
cd ~/
mkdir app
sudo ln -sT ~/app /var/www/html/app
from flask import Flask, request
app = Flask(__name__)

@app.route('/', methods=['POST'])
def receive_file():
    file_content = request.form['data']
    # TODO
    # Implement process to save this data onto a file
    return 'Hello from Flask!'

if __name__ == '__main__':
  app.run()
import sys
sys.path.insert(0, '/var/www/html/app')

from main import app as application
    WSGIDaemonProcess flaskapp threads=5
    WSGIScriptAlias / /var/www/html/app/main.wsgi

    <Directory app>
            WSGIProcessGroup main
            WSGIApplicationGroup %{GLOBAL}
            Order deny,allow
            Allow from all
    </Directory>
response = requests.post('<INTERNAL_INSTANCE_IP>/', headers=headers, data=data)

return "Request sent to your instance with the data of the object"

Loading data into BigQuery from Cloud Storage using Cloud Function (Alternative to functions?)


By : malune
Date : March 29 2020, 07:55 AM
help you fix your problem If you are inserting files from a .CSV use a batch load job. Asking BigQuery to load a file is a sub-second job, and BigQuery will proceed loading them in an async mode.
These loads are free, and your cloud function won't need to wait for it to finish.

How to trigger cloud composer DAG using cloud function when a file is added to cloud storage using Python 3.7


By : AmazingD
Date : March 29 2020, 07:55 AM
will help you Update - as of September 2019, there now exists a Python example in the docs, and the snippets in the docs can be found on GitHub.
Related Posts Related Posts :
  • Why can't I run a .py file that only has a print function?
  • How can I generate an array of 8 unique numbers in python?
  • How to click on an element identical to another element using selenium in python?
  • UsageError: Line magic function `%tensorflow_version` not found
  • Merge two sorted linked lists - Python implementation - wrong/incomplete output
  • Python: proper list comprehension syntax and how to not end up with a list of booleans
  • Why naked Python decorators (without @) do not generate compiler error?
  • Other ways to remove commas in a row of a given column
  • Program written with asyncio takes same time to run as the synchronous version
  • RNN : understanfingConcatenating layers
  • Convert DatetimeWithNanoseconds to date format in python firestore
  • Estimating pi with a Monte Carlo method results in a larger value than expected
  • Create subsets from Python list with suffix
  • How to merge first part of a list value prior to a character, based on the values after the character in python
  • Python, finding item in a list with variables
  • remove demarcation from text image - image processing
  • Cannot create a new environment or install libraries in Ananconda
  • Finding whether a sentence is positive, neutral or negative?
  • Reading license plate from image using OpenCV Python and Tesseract
  • how to run odoo combuted field on all records?
  • path is not reading unless it is in order
  • How to Create a DataFrame with Loops?
  • Displaying the occurrences in a Histogram with tkinter
  • Simulate Fourier Analysis with Python
  • Is there an elegant way to call a list of methods on the same object in Python?
  • When do we need socket and when do we need request?
  • Google Foobar challenge Power Hungry - failing test no. 3 [hidden] out of 5 test cases?
  • Pandas vs Numpy indexing: Why this fundamental difference in ordering of indices?
  • ImportError: cannot import name 'mylist' from 'mymodule'
  • Minesweeper: Reveal Surrounding Blocks Function Freezes
  • Python iterate through connected components in grayscale image
  • LinkedList delete functionality is deleting 2 nodes
  • Minesweeper Reveal Element Python
  • Discord.py bot not using proper text formatting in embeds, but only on mobile?
  • str() doesn't work in the body of my function
  • How to search a list with nested dictionary by dictionary value, returning the index of the list with the dictionary ele
  • Python requests, get() without a protocol?
  • How to run a Python code inside a custom package?
  • Can we filter dictionary based on 'id' in python?
  • Normalization before clustering
  • How to define the name of the model in a function
  • PyTorch CNN: Loss is unchanging
  • Plot square signal from binary array [1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1]?
  • Something is turning my lambda function into def function when I run, how do i solve this?
  • Redisearch aggregate return top 5 of each group
  • How can i save a list/matrix in binary format in KDB?
  • Python coding for GPU
  • Extract value of name in pandas
  • How to get the value of a key value pair from a dictonary which is inside a list and where key starts with a very specif
  • Best place to work together on a python project?
  • DynamoDB - avoid data overwrite with primary partition key remaining the same for all data points
  • Appending (not replacing) items in a nested dictionary via a list of keys
  • Flask Post is not coming to the Same Page
  • TypeError: cannot concatenate 'str' and 'tuple' objects in python 2.7
  • How to split a field within Python and return value in another column?
  • Tkinter + matplot animation filling memory
  • Is it possible for a python file to write another python file?
  • How to use cumsum function group wise in a loop using python
  • Tweepy StreamListener: Tweet when a specified account Tweets
  • Arrange and find most appeared item in the list
  • shadow
    Privacy Policy - Terms - Contact Us © 35dp-dentalpractice.co.uk