Summarize Text with PaLM API Firebase Extension

Photo by Priscilla Du Preez 🇨🇦 on Unsplash

Artificial intelligence is all over the news, with new emergent applications of this technology popping in daily, from image recognition to help diagnose certain health conditions to clever AI chatbots that aid students in their classes. With the release of the Palm APIs from Google, we now have a powerful suite of tools that we can add to our apps, and thankfully, they are already available for integration in Firebase via Firebase Extensions.

Although we may be excited to add AI to every corner of our new and existing applications, we might want to reflect first on why we want to add it and what problem we will solve. Ideally, our products will (most of the time) solve a real-world problem and not be a demo for a new hot piece of technology, so before we can continue, let’s see if we find a new problem to solve and afterward validate if the new PaLM API is a good fit for it.

Our Problem - Reviewing CFP Proposals

For a tech event organizer, besides finding a venue, getting sponsors, and having a good marketing campaign, the most important thing is to find exciting speakers with novel topics that entice the audience. To do so, a Call For Papers (CFP) is created, where everyone can send a proposal for a new talk with a title and a description. This is an easy enough process for small events, where there are about 20 to 30 proposals. However, this can be much more difficult if an event has 100s of proposals. To make matters worse, the proposal for the talk may have a minimum number of characters, making it very time-consuming to review just one of them. The easiest solution for this problem would be to delegate to someone else to create a summary for each application received so that they can be easily skewed to find the most exciting topics. But, this has two drawbacks - it can be expensive to hire someone, and depending on the number of applications, it can be very time-consuming.

At this point, we already know our primary problem - we need to find a cost and time-efficient way to summarize talk descriptions, and with this definition, we can now look into how technology can help us fix it. We need something that can read and summarize an incoming stream of data which are the new proposals. But we also need an application that lets applicants send their proposals and for organizers to review them, which we will create by using Flutter and Firebase, with Firestore, Authentication, and Hosting.

Applicant’s Portal

Organizer’s Portal

This project is currently hosted in Zapp.run, so we can verify its source code and run it there.

We can also look into our model definition since we know all the data we need now - the speaker’s information and their proposal. Since we know we may have a talk summary in the future, we can add that field as nullable.

import 'package:json_annotation/json_annotation.dart';

part 'proposal.g.dart';

@JsonSerializable()
class Proposal {
  final String speakerName;
  final String speakerEmail;
  final String talkTitle;
  final String talkDescription;
  final String? talkSummary;

  Proposal({
    required this.speakerName,
    required this.speakerEmail,
    required this.talkTitle,
    required this.talkDescription,
    this.talkSummary,
  });

  factory Proposal.fromJson(Map<String, dynamic> json) =>
      _$ProposalFromJson(json);
      
  Map<String, dynamic> toJson() => _$ProposalToJson(this);
}

Finding a Firebase Extension for our needs

When we look into our data on Firestore, we see a new button at the top saying “Extensions - NEW,” one of the points of entry to the Firebase Extensions Hub. In summary, it’s a marketplace with ready-made solutions that we can easily add to our app to supercharge them with chatbots, image optimization, integration of 3rd party services, or AI tools.

This would be a great place to start searching for a solution to our problem since it would let us skip implementing an AI tool that would look into our Firestore data and pick the correct data to summarize. Thankfully, in the new AI section, we see an Extension that fits our needs - Summarize Text with PaLM API.

On the detail page, we find and review the essential information about this Extension, including a brief explanation of Getting Started, all the parameters we can configure for this extension, the Firebase and GCP resources created, an installation guide, and billing information.

Installing a Firebase Extension

We can click the big “Install in Firebase console” button to install it.

This shows us a list of our current Firebase projects, where we can select the project (or create one) on which we want to install the extension.

As with other Firebase Extensions, before we install it, it asks us to upgrade to the Blaze Plan by associating a billing account.

Next, we can review all the resources that will be created alongside a list of all the necessary APIs we need to enable. We can review this second list, and click “Enable” if one of the APIs is missing from our project.

Following that, we can review all the permissions necessary for the project and the different Service Accounts that will be created to access the different resources.

Finally, we can configure the Extension in the last step. Here we will be able to select the Cloud Function location and the API Provider we will use. If we don’t have access to PaLM API, we can use Vertex AI to use this Firebase Extension.

We will also configure the necessary parameters for our Firestore data:

• The Collection Name where we have our data;
• The Text Field that we want to summarize;
• The Response Field that is going to be created with the output;
• The Target Summary optional field states the “Number of sentences you would like the summary to be,” an integer between 1 and 10.

With all this information, we can click on the “Install extension” button.

This will show us a new page, within our Firebase Console, where we can see the current state of the installation and a list of all other installed Firebase Extensions.

With the Extension installed, we can test our application by sending a new proposal. To send a proposal, we can use the cloud_firestore package and use our Proposal class to get the necessary information to add a new document to the proposals collection:

class ProposalProvider {
  final FirebaseFirestore firestore;

  const ProposalProvider(this.firestore);

  Future<void> sendProposal(Proposal proposal) async {
    try {
      await firestore.collection('proposals').add(
            proposal.toJson()..removeWhere((key, value) => value == null),
          );
    } catch (e) {
      debugPrint('Error while sending proposal: $e');
      rethrow;
    }
  }
}

When we check Firestore, we will see that our document was updated with several new fields, including safetyMetadata, status, which can help us debug in case there was an error with the Extension and the field we specified in the Extension’s configuration - talkSummary.

{
  "speakerEmail": "solid.goncalo@gmail.com,"
  "speakerName": "Gonçalo Palma",
  "safetyMetadata": {
    "blocked": false,
    "safetyAttributes": {
      "categories": [
        "Death",
        "Harm & Tragedy",
        "Health"
      ],
      "blocked": false,
      "scores": [
        0.1,
        0.4
      ]
    }
  },
  "status": {
    "state": "COMPLETED",
    "completeTime": 1692615135,
    "updateTime": 1692615135,
    "error": null,
    "startTime": 1692615133
  },
  "talkSummary": "Emma is a developer who used Flutter to create an app called HomeBound.\nHomeBound uses AR to bring families and friends closer, even when they are continents apart. Emma's grandmother was sad every Christmas because she missed her childhood home in Italy.\nEmma used Flutter to create an AR app that allowed her grandmother to virtually travel back to her childhood home.\nHomeBound became an overnight sensation and people from all over the world started using it to bridge the gap between them and their loved",
  "talkDescription": "...",
  "talkTitle": "Emma's journey in Flutter"
}

We can now use the cloud_firestore package to list all the documents from the proposals collection to see our new data.

class ProposalProvider {
  final FirebaseFirestore firestore;

  const ProposalProvider(this.firestore);

  Future<List<Proposal>> listProposals() async {
    try {
      return firestore
          .collection('proposals')
          .get()
          .then((ref) => ref.docs)
          .then(
            (docs) => docs.map((doc) {
              try {
                print(jsonEncode(doc.data()));
              } catch (e) {
                print(e);
              }

              return Proposal.fromJson(doc.data());
            }).toList(),
          );
    } catch (e) {
      debugPrint('Error while sending proposal: $e');
      rethrow;
    }
  }
}

Adding security to our application

For a proof-of-concept application, this is where our application would be finished. However, to publish this app publicly, we must find ways to protect our data. For our problem definition, only Applicants can create a new proposal, with no option to edit them, and only the Firebase Extension can edit a proposal by adding a new field. These restrictions can be added through Firestore Rules.

Let’s start with the easiest part - a user can only create and read proposals (in reality, we should add a new Custom Claim for an admin in order not to allow users to read proposals, but that will be a topic for another article, stay tuned!), on the proposals collection. To do so, we must ensure that they are authenticated.

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /proposals/{proposal} {
      // Allow authenticated users to create and read documents
      allow read, create: if request.auth != null;
  }
}

Next, we know that we want the Firebase extension to update one of the fields, not the whole document, to mitigate any possible errors with the application. However, we have a minor issue - how can we identify, in the rules, that we want only the Firebase Extension to update the talkSummary field and no one else?

If we go back to the installation process of our Extension, we will see that some Service Accounts were created. We can review them by visiting our Firebase Console > Settings > Service Accounts.

Here we will have a list of the different Service Accounts for Firebase and a link for “All service accounts” that opens the GCP console.

On this new page, we see our Extension’s Service account as the first element of the list (evident by its name and the prefix on the “Principal”). With this email, we can create a new Firebase Rule that verifies if this email address is sending the request.

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /proposals/{proposal} {
      // Allow authenticated users to create and read documents
      allow read, create: if request.auth != null;
      
      // Allow only the service account to update the talkSummary field
      allow update: if request.auth.token.email == 'service-account-email@your-project-id.iam.gserviceaccount.com'
                    && request.resource.data.keys().hasOnly(['talkSummary']);
    }
  }
}

With this, we ensure that we only give the required permissions to every user and service that uses our application.

Conclusion

Once again, Firebase Extensions were an invaluable tool to kickstart our new project. It allowed us to ideate and integrate a third-party service in our app with minimal configuration. Within minutes, we had a powerful AI model summarizing parts of our application without the need to read documentation, create new Cloud Functions, set up permissions, roles, and all the other necessary work. Not only that, but it was easy to review all the resources created so that we could protect our database with Firestore Rules.

Now, it’s your turn to find a problem to solve and see if there is a Firebase Extension that can help you build it in record time.

Originally published at Invertase - Summarize Text with PaLM API Firebase Extension.