Firestore Integration

Agenty Firebase workflow allows you to send your web scraping agent’s result to Google Cloud Firebase Firestore NoSQL database by connecting to your project id, collection using service credentials.

Firebase is a real-time NoSQL database built on Google cloud infrastructure to scale automatically.

The Agenty Firebase workflow allows you to import your agent’s result to selected Firestore project database collection automatically.

Create Service Account

Agenty requires the service credential JSON file to connect with your Google cloud services. So, the first step to start using Firestore plugin in Agenty is to get your service credential file.

If you have the key file already with Firestore permission? You can skip this step and jump to configure the workflow directly.

A service account represents a google cloud service identity, such as code running on Compute Engine VMs, Server-less etc.

1. Login to your Google cloud account and go to IAM & Admin page
2. Click on the Service Accounts in left menu to go to service account page
   
   

   google cloud platform858×648 41.8 KB
3. Click on the Create Service Account button to create a new service account. Then enter the name and description of your choice.
   
   

   Create Google cloud service account1027×700 51.7 KB
4. Add a role to give full read and write access to Firestore Database

Add a Firestore role982×642 45.7 KB

Create Firestore key1015×912 68.1 KB

Create service credential key1066×335 33.1 KB

Configure Firebase Workflow

1. Login to your Agenty account
2. Go to the Workflows tab

Create a workflow1546×386 25.3 KB

3. Click on the +Create a workflow button

Attach Firebase workflow892×764 64.7 KB

4. Firebase plugin configuration page will be open
5. Select the agent where you want to attach this workflow to trigger on job completion.

Select the agent for Firebase integration890×448 30.2 KB

6. Then upload the credentials file and enter the project id, collection name in workflow configuration

Firebase workflow configuration921×775 70 KB

7. The primary key field is optional. This can be used to specify a primary key field from your agent to overwrite existing documents (if any). If not specified, Firestore will auto-generate primary key to insert as new document
8. Save this workflow configuration to attach it with the selected agent

When a primary key field is specified, Agenty will use that field value and send an overwrite request to Firestore. If the document does not exist, it will be created. If the document does exist, its contents will be overwritten with the newly provided data

Execution

Go to the agent page where you’ve attached this Firebase workflow. For example, I attached it to a web scraping agent with name : Firestore plugin test

And, start that agent. (You may also start the web scraping task using our API)

It will take a few seconds to complete the job, depending on the number of URLs you are crawling in this agent. So, wait for the completion.

web scraping result to insert on Firestore1325×699 55.9 KB

Logs and Preview

Click on the logs tab and scroll to the last lines to see the plugin execution logs. You will see the message with Added or Updated document with id successfully.

Go to your Firestore project page - https://console.firebase.google.com/ and select your project. Then go to database > collection to see the result sent by Agenty.

Firestore database preview1575×778 78.3 KB