Overview

Everyone is talking about AI. Many leaders in the public sector have been tasked with finding ways to use AI to solve our day-to-day challenges.

It’s really hard to do that, however, without knowing what AI technologies are capable of. There is a common misconception that AI is a swiss-army knife technology, where one “AI” can be applied to most problems. The truth is that there are several different kinds of AI technologies out there, and each technology is as different from the other as a hammer is from a screwdriver.

This blog post will be the first in a series of blog posts designed to help municipality leadership understand the different kind of AI technologies and how to use them appropriately.

In this particular blog post, we’ll break down:

• What is AI?
• How to Choose the Right AI Model for your Use-Case
• The differences between 6 Common Kinds of AI Technologies
• For each AI Technology
  • Key Uses
  • Applications in Government
  • Additional Considerations

What is AI? A Simple Definition of Artificial Intelligence

Artificial Intelligence, put simply, is intelligence demonstrated by machines. Artificial Intelligence researchers study and develop methods and models/algorithms to imbue machines with the power to mimic “intelligence”. This means that many technologies we use today can be considered “AI” (search engines, spell-check, the built-in chess application on our computers, etc.).

Tasks for Artificial Intelligence Technologies

Artificial Intelligence technologies generally perform one of 3 tasks.

• Classify
• Predict / Calculate a Numeric Value
• Generate

Classification AI technologies are AI technologies that are good at predicting the category or class a situation, document, etc. should fall into.‍

Value-Predicting AI technologies are AI technologies that perform regression techniques or simulations to determine a numeric output based on some input.‍

Generative AI technologies (or GenAI for short) are AI technologies that excel in tasks that require the creation of text or images, typically based on some prompt or source material.

Fun Fact: At their core - generative technologies are actually Classification AI technologies. For example, in a large-language model like ChatGPT, each word is treated as a “category” and the model is essentially outputting the most likely “category” (e.g. word) to follow a given input prompt, but we will explain that further in a future blog...

Neural Networks: The key to new AI technologies

The difference between many of the older AI technologies and newer AI technologies is that many newer AI technologies leverage a special kind of model called a “neural network”. We’ll discuss how neural networks work in a future post.

In this post, we’ll summarize different kinds of neural network (NN) and non-NN technologies, and how to use each one appropriately.

‍

How to Choose the Right AI Model for your Use-Case

When choosing AI models to solve a problem, you’ll need to know the following 3 things:

1. Data Input (e.g. text, images, series of numbers): You should know what type of input the model will be expected to receive. Will it be text? An image?
2. Data Output (e.g. text, images, series of numbers): Similarly, you should know what type of output the model will be expected to provide to your user.‍
3. Type of Task (e.g. classification, numeric value-prediction, or generation): You should know whether the task is one that requires classification, numeric value prediction, or generation of content.

The ideal choice of AI technology will depend on these three factors. In this blog, we will outline these factors for 6 common kinds of AI technologies.  The 6 technologies are:

1. Large Language Models (LLMs) (Generation)
2. Retrieval Augmented Generation (RAG) (Generation)
3. Generative Adversarial Network (GAN) for Images (Generation)
4. Computer Vision Models (Classification)
5. Rule-Based/Decision-Tree Models (Classification)‍
6. Optimization & Simulation Models (Value Prediction)

We’ll also outline the common government use-cases for each kind of technology.

6 Common Kinds of Artificial Intelligence Technologies

Large Language Models (LLM)

Data Input: Text/Words
Data Output: Text/Words
Type of Task: Generation

Key Uses:

• Summarizing & re-phrasing text
• Expanding upon text
• Completing fill in the blank exercises
• Translation of languages

Applications in Government:

• Drafting proposals for grants, drafting documentation, job descriptions, etc.
• Summarizing legislation when doing policy research
• Providing draft translations of public content to improve accessibility

Considerations:

• No understanding of generated content
• Hallucination

Examples:  ChatGPT, Gemini, Claude, etc.

How it Works: The question that a Large Language Model is constantly answering is: “Given this set of previous words, what is most-likely word from the dictionary to fill in the blank/ show up next?”. It turns out that when you’ve learned patterns of language from the entire internet, you can develop a pretty good sense of what words tend to follow what other words.

‍

‍

Retrieval Augmented Generation (RAG)

Data Input: Text/Words
Data Output: Text/Words
Type of Task: Generation

Key Uses:

• Answering questions based on a specified knowledge base  (e.g. Chatbot applications)

Applications in Government:

• Chatbots for answering citizen questions about facts of a new programs, information on a website, how to fill out forms, etc.
• Help city employees navigate city offered benefits

Considerations:

• No understanding of generated content
• Requires a list of documents to create a knowledge base
• The effectiveness of a RAG solution depends highly on how well you’re able to identify the relevant documents from the knowledge base for a particular query/prompt

Examples: Perplexity (AI powered internet search engine)

How it Works: A RAG-based technology is actually a combination of two other technologies: a search engine and an LLM. After the prompt from the user is received, the search engine/ranking algorithm (like what Google uses) is used to determine the most relevant documents in the knowledge base for the given prompt. Then the prompt AND the relevant documents are passed into an LLM like ChatGPT to provide a more-accurate, but still user-friendly, chat-bot like response.

Generative Adversarial Network (GAN) for Images

Data Input: Text, Image (most of the time)
Data Output: Image
Type of Task: Generation

Key Uses:

• Generating images that look real

Applications in Government:

• Putting together conceptual images for proposals (e.g. Helping communities envision alternative uses of a property/ city land for parks, public works, art, etc.)

Considerations:

• Can only generate images similar to the “Real Images” that it has been trained on

Examples: DALLE-3, Adobe Firefly, Microsoft Copilot, Midjourney

How it Works: Generative AI Models for images are models that have been trained to output grids of pixel values that, to our human eye, look like coherent images. In the training process, the “Generator Model” is asked to produce an output that looks real. The output of the Generator Model is given to a “Discriminator Model” alongside real-life images. The Discriminator Model is asked to determine which images are real and which are fake/generated by the Generator Model. The two models are trained to compete with one another. Through the training process, Discriminator Model gets better at determining which images are fake, while the Generator Model gets better at producing images that can fool the Discriminator.

‍

Computer Vision Models

Data Input: Text, Image (most of the time)
Data Output: Category Label (Text)
Type of Task: Classification

Key Uses:

• Pattern Matching (finding parts of an image that look similar to a provided image)
• Classifying the kind of image: What kind of thing is this?
• Image Detection (finding the bounding boxes for parts of an image that fit in a particular category)
• Digitization of Paper-based Government Forms

Applications in Government

• Finding relevant information in government documents (e.g. permit plan review, front-desk/counter work)
• Evaluating environmental conditions of natural resources over time (like coastline erosion)
• Facial Recognition for Security  (e.g. like TSA Pre-Check/Airport Security)
• Evaluating the conditions of public infrastructure assets like bridges and sewage pipes using drone-captured images

Considerations:

• Need millions of sample image to train neural network-based computer vision models
• Non-neural network approaches do not work well with noisy images (e.g. less-than-perfect conditions in the image like overlapping images/text, blurriness, etc.)

Examples: OCR in Adobe Acrobat (the ability to search for and find text in PDFs), Facial Recognition on iPhones

How It Works: Computer Vision models are trying to answer the question “What category of object does this pixel belong to?”. Typically, there is a pattern image which is representative of the category/ image you are trying to detect. The pattern image is overlaid on top of the image of interest (”base image”), and a mathematical function (called a “convolution”) is used to determine how much overlap there is between the pattern image and the underlying portion of the base image. Pixels in areas of high overlap/ above a certain “threshold” of overlap are assumed to match the pattern image and belong to the relevant category.

The difference between the neural network approach and the non-neural network approach is whether you know the pattern image. In a neural network approach, a convolutional neural network (CNN) is used to learn different “pattern images” for each category. In a non-neural network approach, the “pattern images” are pre-defined by the software creator.

‍

‍

Rule-Based/ Decision Tree Models

Data Input: Series of Numbers (typically), Text, Images
Data Output:  Category Label/Decision (Text or Number)
Type of Task:  Classification

Key Uses:

• You have a list of known rules and want to evaluate known information against the rules
• You have a clear decision tree/ routing structure, which you want to use to make a decision

Applications in Government:

• Driving permit exams
• Automated routing of documentation through government process workflows
• Permit & License issuance
• Guided government document/form submission and completion checks

Considerations:

• Requires a clear understanding of the rules/decision tree
• Takes a lot of time and expertise to translate knowledge of the rules into computer-understandable rules
• Time-consuming changes when changing processes, which result in changes to the rules / decision trees

Examples:  IRS Free File, TurboTax

How It Works: Rules-based systems apply a set of predefined rules to process information and make decisions. They have an engine that uses an established rule base and inference to determine appropriate actions or outcomes based on the given input.

‍

Optimization & Simulation Models

Data Input: Series of Numbers (typically), Images
Data Output: Number
Type of Task:  Numeric-Value Prediction

Key Uses:

• Simulating a known set of interactions (e.g. traffic patterns, weather patterns)
• Optimization Problems (e.g. trying to minimize or maximize a value - knowing the different interactions at play in a situation)

Applications in Government:

• Simulating road traffic
• Simulating environmental impact of civic projects with digital twining 
• Scheduling optimization for meetings
• Optimizing budget allocations based on predicted and past usage data

Considerations:

• Limited to simulating interactions that a human being can fully-understand (Modeling complicated dynamics that humans do not understand can lead to bad approximations with room for error)
• Optimization models only work well as the simulation/interaction model they are paired with

Examples: Road Traffic Simulation Software, Urban Planning Modeling Software

How It Works: Optimization models are trying to find the best values for certain parameters that either maximize or minimize a specific objective, such as cost or efficiency, subject to certain constraints. The constraints are often modeled using a simulation model, which mimics the behavior of real-world systems to predict outcomes under different scenarios.

‍

‍

In Summary

Understanding the diverse capabilities and appropriate applications of AI technologies is essential for public sector leaders looking to use these tools effectively. By recognizing the unique functions of different AI models, leaders can make informed decisions that address specific challenges.

Next steps: We encourage you to explore each AI technology discussed in this post further. In our upcoming series, we'll dive deeper into these technologies, offering practical examples and case studies to demonstrate their impact in government settings. Stay tuned for more insights and strategies to successfully implement AI in your organization.

‍

‍