Everyday Llamas

If you work at a company with protected IP (intellectual property), you know there’s a risk in using remote LLMs (large language models): You cannot be certain whom you’re sharing your company’s code with and what they’ll use it for.

If you didn’t already know, I’m so glad you do now. 😌

Nevertheless, LLMs can help speed up daily tasks. They are new and exciting and naturally spark most engineers curiosity. While we shouldn’t see them as a solution for every problem, they can be great for a number of repetitive tasks.

For example, tasks where pattern recognition or repetition are essential.

You might be unaware of the diff tool and how to use it, but if you’re familiar with Copilot (which is hugely popular right now), you know that you can use it to ask questions about your code. You could ask about differences in your file.

You might also know that you can upload files to ChatGPT and ask questions about them.

Yes, we’ve all been there, pasting a snippet followed by WHY THIS ISN’T WORKING!!?? 😅

It’s important to know that you may be sharing your company’s IP to which authorization is required. If your company has cleared you to share code, that’s great. If they haven’t, here’s a safer alternative.

Wouldn’t it be cool to have a tool that solves these problems and does so without sharing your valuable code?

What if it looked like…

~ % gemma "Summarize the differences between these two files: $(cat 1.txt) and $(cat 2.txt)"
The primary difference between "Hello" and "Hello!" is the exclamation point at the end of "Hello!".

Let’s build it (or an elementary and contrived version 😉).

We’ll explore using Ollama locally without sending your code to an external service.

If you’re using Homebrew, you can:

% brew install ollama

You might also need to start the ollama service:

% brew services start ollama

Once you have Ollama installed, you’ll need to download a model. I’ve been enjoying Google’s Gemma 2 model lately, which we’ll pull to use in this demo.

% ollama pull gemma2

Now, you should be able to run the model using ollama run gemma2:

>>> Hello!
Hello! 👋 How can I help you today? 😄

Now, here’s the fun part:

Another way of using ollama is by providing your prompt to the command line application:

% ollama run gemma2 "Tell me a joke"
Why don't scientists trust atoms?

Because they make up everything! 😊

If the command-line app accepts parameters, we could pipe or interpolate anything from the filesystem, just like any other command-line app.

Let’s create an alias to keep things shorter:

% alias gemma="ollama run gemma2"

We’re ready to ask questions.

% gemma "Which database is this application using? $(cat Gemfile)"
Based on the provided Gemfile, the application uses **PostgreSQL** as its database.

Here's why:

* The line `gem "pg"` explicitly includes the PostgreSQL driver for Ruby.

This allows us to do fun things like automating commit messages based on the actual code you just wrote¹ and without sharing company secrets!

For example, if we add the ruby-lsp gem:

diff --git a/Gemfile b/Gemfile
index 87325jfo0a..d6v31j8q5 100644
--- a/Gemfile
+++ b/Gemfile
@@ -108,6 +108,7 @@ group :development do
   gem "rubocop-performance"
   gem "rubocop-rails"
+  gem "ruby-lsp"
   gem "web-console"
 end

We can:

% gemma "Write a commit message for this changeset: $(git diff --cached)"
Add ruby-lsp for language server support

This commit adds the `ruby-lsp` gem to the development group of the Gemfile.
This will enable Language Server Protocol (LSP) support in editors like VS
Code, providing features such as code completion, diagnostics, and
refactoring.

Neat, isn’t it?

The sky is the limit… or maybe we should say the cloud is the limit.

You could pipe this in turn to pbcopy to paste its output or directly to git commit -m to do it all at once.

Have fun!