How to Fill Gemini's 2 Million Token Context Window with Code

When I heard that Google Gemini’s GPT service had a 2 million token context window, the first thing I thought was: I wonder if I can upload my entire src directory?

Unfortunately, when writing this, the Gemini web console allows only uploading images, not source code files.

But if you use the cloving cli tool, you can quickly start prompting your entire source code repository with Gemini.

How to Generate Huge Prompts

Instead of manually copy/pasting large numbers of files, the cloving cli will stitch together a prompt for you that includes any and all files from a repository that you would like:

npm install cloving -g # install cloving cli globally
cloving config # configure your AI API keys/tokens

cd myapp # cd into your project directory
cloving init # analyze your project

cloving generate review --model gemini \
  --files src \
  --prompt "Which code files need to be refactored the most?"

This will recursively look into every file and subdirectory of src’s files and generate a comprehensive prompt that looks something like this:

### Prompt

Which code files need to be refactored the most?

### Description of App

{
  "languages": [
    {
      "name": "TypeScript",
      "version": "^5.1.3",
      "primary": true
    }
  ]
}

### Contents of src/App.vue

<template>
  <div class="application">
    etc. etc.

### Contents of src/components/AppOverview.vue

<template>
  <div class="app-overview">
    <div class="overview-content">
      etc. etc.

... MORE CONTENTS FROM src ...

### Request
        
I would like you to explain the code and document a description of it.
List any bugs in the new code as well as recommended fixes for those bugs with code examples.
Format the output of this code review in Markdown format.

### Additional Context
            
Which code files need to be refactored the most?

Before sending this to the AI API endpoint, you can review and edit this prompt as you see fit in your favorite text editor. Just set the EDITOR environmental variable to vi or emacs or pico or nano or even code --wait for VS Code.

The more code, the bigger your prompt

This prompt will be sent to any AI API you want, but not every AI API can handle large context windows.

Here is a summary of the context window lengths for some of the models currently supported by cloving:

Claude Models

• Claude 3.5 (sonnet-20240620): 200k tokens
• Claude 3 (opus-20240229): 200k tokens
• Claude 3 (sonnet-20240229): 100k tokens
• Claude 3 (haiku-20240307): 16k tokens

Gemini Models

• Gemini 1.5 (pro-latest): 2m tokens (or 128k if using free)
• Gemini 1.5 (flash-latest): 1m tokens
• Gemini 1.0 (pro-latest): 32k tokens

Mistral Models

• Mistral (embed): 8k tokens
• Mistral (tiny-latest): 8k tokens
• Mistral (small-latest): 32k tokens
• Mistral (medium-latest): 64k tokens
• Mistral (large-latest): 128k tokens
• Open Mistral (7b): 8k tokens
• Open Mixtral (8x7b): 32k tokens
• Open Mixtral (8x22b): 64k tokens
• Codestral (latest): 128k tokens

Ollama Models

• Llama 3 (70b-instruct): 16k tokens
• Llama 3.1 (70b): 8k tokens
• Llama 3.1 (latest): 16k tokens
• DeepSeek Coder V2 (latest): 64k tokens
• Codestral (22b): 64k tokens

OpenAI Models

• GPT-4o: 128k tokens
• GPT-4o-mini: 64k tokens
• GPT-4 Turbo: 128k tokens
• GPT-3.5 Turbo: 16k tokens
• Text-embedding (3-large): 8192 tokens

Generating Code with Large Prompts

So far, we have seen the cloving cli do a code review on an entire codebase by generating code, but you can also generate code with the cloving cli.

npm install cloving -g # install cloving cli globally
cloving config # configure your AI API keys/tokens

cd myapp # cd into your project directory
cloving init # analyze your project

cloving generate code --model gemini \
  --files src \
  --prompt "Refactor src/components/AppOverview.vue into separate components" \
  --save

This will generate a prompt like this:

### Request

Generate code that does the following: Refactor src/components/AppOverview.vue into separate components

### Description of App

{
  "languages": [
    {
      "name": "TypeScript",
      "version": "^5.1.3",
      "primary": true
    },
    {
      "name": "JavaScript",
      "version": "ES6+"
    }
  ]
}

### Contents of package.json

{
  "name": "myapp",
  "version": "1.2.8",
...
}

### Contents of src/App.vue

<template>
  <div class="application">
    etc. etc.

### Contents of src/components/AppOverview.vue

<template>
  <div class="app-overview">
    <div class="overview-content">
      etc. etc.

... MORE CONTENTS FROM src ...

### Request
  
Generate code that does the following: Refactor src/components/AppOverview.vue into separate components

It will then send the request to the AI API and automatically save the generated code into your codebase for you to review in detail.

Conclusion

The cloving CLI tool revolutionizes how developers interact with AI models by enabling the seamless integration of entire codebases into prompts.

With the ability to handle large context windows, such as the impressive 2 million tokens offered by Google Gemini, cloving makes it feasible to conduct comprehensive code reviews and generate code refactoring suggestions across extensive projects.

By automating the prompt generation process and supporting various AI models, cloving empowers developers to leverage AI capabilities efficiently, saving time and enhancing productivity.

Whether you’re looking to review your codebase or generate new code, cloving provides a powerful and user-friendly solution to maximize the potential of AI in software development.