Open-source language models have transformed coding and content creation. Running these open source LLMs locally offers privacy, control, and no API fees. Whether for code LLM programming or writing model creative work, this guide covers top open source language models today.

This list highlights popular open-source LLMs for coding and writing, organized by recency and popularity. Each model entry shows key attributes, use cases, tags, and pros and cons. These models work with tools like Ollama, LM Studio, Jan, and other local AI platforms.

Review the How to Choose the Right Model section for selection guidance.

⚠️ Hardware requirements vary by model size. Smaller models (1B-7B) can run on CPU-only systems with modest RAM, while larger models require significant GPU or CPU resources.

gpt-oss

Tags: writing, coding

Owner/Author: OpenAI

Parameters: 20B (3.6B active), 120B (5.1B active)

Resource Demand:High (20B), Very-High (120B)

Primary Use Cases: Reasoning, agentic tasks, function calling, structured outputs, tool use, general-purpose tasks

Pros:

  • OpenAI's open-weight models with state-of-the-art reasoning
  • Mixture-of-Experts (MoE) architecture for efficiency
  • 128K context window for long-context tasks
  • Native support for function calling and structured outputs
  • Adjustable reasoning effort (low, medium, high)
  • Full chain-of-thought access for debugging
  • Apache 2.0 license
  • OpenAI-compatible API

Cons:

  • 20B model requires ~16GB VRAM (14GB download)
  • 120B model requires ~60GB VRAM (65GB download)
  • Very new models with less community testing
  • Large download sizes
  • May be overkill for simple tasks

qwen3-coder

Tags: coding

Owner/Author: Alibaba Cloud (Qwen)

Parameters: 30B, 480B

Resource Demand:High (30B), Very-High (480B)

Primary Use Cases: Agentic coding tasks, long-context code generation, complex coding scenarios

Pros:

  • Excellent performance on long-context coding tasks
  • Strong support for agentic workflows
  • Cloud deployment options available
  • Tools integration support

Cons:

  • Very large model sizes (especially 480B) require significant resources
  • Relatively new (3 months old) with less community testing
  • May be overkill for simple coding tasks

qwen2.5-coder

Tags: coding

Owner/Author: Alibaba Cloud (Qwen)

Parameters: 0.5B, 1.5B, 3B, 7B, 14B, 32B

Resource Demand:Low (0.5B-7B), Medium (14B), High (32B)

Primary Use Cases: Code generation, code reasoning, code fixing, general programming assistance

Pros:

  • Wide range of model sizes for different hardware constraints
  • Significant improvements in code generation and reasoning
  • Most popular code model (10.1M pulls)
  • Tools integration support
  • Excellent code fixing capabilities

Cons:

  • Larger models (32B) still require substantial resources
  • Smaller models (0.5B, 1.5B) may lack depth for complex tasks
  • Newer model with less long-term reliability data

deepseek-coder-v2

Tags: coding

Owner/Author: DeepSeek

Parameters: 16B, 236B

Resource Demand:Medium (16B), Very-High (236B)

Primary Use Cases: Code-specific tasks, performance comparable to GPT-4 Turbo

Pros:

  • Mixture-of-Experts architecture for efficiency
  • Performance comparable to GPT-4 Turbo on code tasks
  • Strong code generation quality
  • Well-optimized for code-specific scenarios

Cons:

  • 236B model requires extremely high-end hardware
  • Smaller 16B model may not match larger variants
  • Less general-purpose than some alternatives

codestral

Tags: coding

Owner/Author: Mistral AI

Parameters: 22B

Resource Demand:Medium (22B)

Primary Use Cases: Code generation tasks, Mistral AI's specialized code model

Pros:

  • First code model from Mistral AI (reputable developer)
  • Good balance between size and performance
  • Strong code generation capabilities
  • Well-maintained and supported

Cons:

  • Single size option limits flexibility
  • Relatively new (1 year old)
  • May not match performance of larger specialized models

deepseek-coder

Tags: coding

Owner/Author: DeepSeek

Parameters: 1.3B, 6.7B, 33B

Resource Demand:Low (1.3B-6.7B), High (33B)

Primary Use Cases: General code generation, trained on 2 trillion tokens

Pros:

  • Extensive training on 2 trillion code and natural language tokens
  • Multiple size options for different use cases
  • Very popular (2.7M pulls)
  • Strong general-purpose coding capabilities
  • Well-tested and reliable

Cons:

  • Older model (2 years) may lack latest improvements
  • 33B model requires significant resources
  • Smaller models may lack depth for complex reasoning

codellama

Tags: coding

Owner/Author: Meta

Parameters: 7B, 13B, 34B, 70B

Resource Demand:Low (7B), Medium (13B), High (34B-70B)

Primary Use Cases: Text-to-code generation, code discussion, general programming

Pros:

  • Extremely popular (4M pulls)
  • Multiple size options including very large 70B model
  • Strong general-purpose capabilities
  • Can discuss and explain code, not just generate
  • Well-established and reliable

Cons:

  • 70B model requires very high-end hardware
  • Older architecture compared to newer models
  • May not specialize as well as code-specific models

starcoder2

Tags: coding

Owner/Author: BigCode (Hugging Face)

Parameters: 3B, 7B, 15B

Resource Demand:Low (3B-7B), Medium (15B)

Primary Use Cases: Transparently trained open code LLMs, general code generation

Pros:

  • Transparent training process (open and reproducible)
  • Good range of sizes for different hardware
  • Strong code generation capabilities
  • Well-documented and community-supported

Cons:

  • May not match performance of newer specialized models
  • Limited to three size options
  • Less specialized than code-specific variants

codegemma

Tags: coding

Owner/Author: Google

Parameters: 2B, 7B

Resource Demand:Low (2B-7B)

Primary Use Cases: Fill-in-the-middle completion, code generation, natural language understanding, mathematical reasoning

Pros:

  • Lightweight models suitable for resource-constrained environments
  • Versatile capabilities beyond just code generation
  • Strong fill-in-the-middle completion
  • Good for mathematical reasoning tasks

Cons:

  • Smaller models may lack depth for complex tasks
  • Limited size options
  • May not match larger models on complex code generation

granite-code

Tags: coding

Owner/Author: IBM

Parameters: 3B, 8B, 20B, 34B

Resource Demand:Low (3B-8B), Medium (20B), High (34B)

Primary Use Cases: Code Intelligence, IBM's open foundation models

Pros:

  • Good range of sizes from small to large
  • IBM-backed with enterprise support
  • Focused on code intelligence tasks
  • Well-maintained foundation models

Cons:

  • Less popular than some alternatives
  • May have IBM-specific optimizations
  • Less community testing compared to more popular models

deepcoder

Tags: coding

Owner/Author: Agentica

Parameters: 1.5B, 14B

Resource Demand:Low (1.5B), Medium (14B)

Primary Use Cases: Code generation at O3-mini level performance

Pros:

  • Fully open-source with transparent development
  • Performance comparable to O3-mini level
  • Good balance with 14B model
  • Lightweight 1.5B option available

Cons:

  • Limited to two size options
  • May not match latest model capabilities
  • Less popular than mainstream alternatives

opencoder

Tags: coding

Owner/Author: OpenCoder Team

Parameters: 1.5B, 8B

Resource Demand:Low (1.5B-8B)

Primary Use Cases: Open and reproducible code LLM, English and Chinese support

Pros:

  • Open and reproducible training process
  • Bilingual support (English and Chinese)
  • Good for international development teams
  • Lightweight options available

Cons:

  • Limited size options
  • Less popular than alternatives
  • May not match performance of larger models

yi-coder

Tags: coding

Owner/Author: 01-ai (Yi)

Parameters: 1.5B, 9B

Resource Demand:Low (1.5B-9B)

Primary Use Cases: State-of-the-art coding performance with fewer parameters

Pros:

  • Efficient performance with fewer parameters
  • Good coding performance relative to size
  • Lightweight options for resource-constrained environments
  • Optimized for coding tasks

Cons:

  • Limited size options
  • May not match larger models on complex tasks
  • Less popular than mainstream alternatives

codegeex4

Tags: coding

Owner/Author: Zhipu AI (CodeGeeX)

Parameters: 9B

Resource Demand:Medium (9B)

Primary Use Cases: AI software development, code completion

Pros:

  • Versatile for various AI software development scenarios
  • Strong code completion capabilities
  • Single optimized size option
  • Good for IDE integration

Cons:

  • Only one size option available
  • May not match performance of larger models
  • Less popular than alternatives

codeqwen

Tags: coding

Owner/Author: Alibaba Cloud (Qwen)

Parameters: 7B

Resource Demand:Low (7B)

Primary Use Cases: Large language model pretrained on extensive code data

Pros:

  • Extensive pretraining on code data
  • Good balance of size and performance
  • Part of Qwen model family
  • Well-optimized for code tasks

Cons:

  • Only one size option
  • May not match latest Qwen2.5-coder improvements
  • Less flexible than multi-size alternatives

dolphincoder

Tags: coding

Owner/Author: Eric Hartford (community)

Parameters: 7B, 15B

Resource Demand:Low (7B), Medium (15B)

Primary Use Cases: Uncensored coding variant, based on StarCoder2

Pros:

  • Uncensored variant for unrestricted coding scenarios
  • Based on proven StarCoder2 architecture
  • Two size options available
  • Good for scenarios requiring fewer restrictions

Cons:

  • Uncensored nature may not be suitable for all use cases
  • Less popular than mainstream alternatives
  • May have ethical considerations for some teams

stable-code

Tags: coding

Owner/Author: Stability AI

Parameters: 3B

Resource Demand:Low (3B)

Primary Use Cases: Code completion, instruction following, coding tasks

Pros:

  • Very lightweight (3B) suitable for most hardware
  • Performance comparable to Code Llama 7B despite smaller size
  • Good for code completion tasks
  • Stable and reliable

Cons:

  • Only one size option
  • May lack depth for complex reasoning tasks
  • Smaller than many alternatives

magicoder

Tags: coding

Owner/Author: iSE-UIUC

Parameters: 7B

Resource Demand:Low (7B)

Primary Use Cases: Code generation trained on 75K synthetic instruction data

Pros:

  • Novel OSS-Instruct training approach
  • Trained on open-source code snippets
  • Good for general code generation
  • Innovative training methodology

Cons:

  • Only one size option
  • Less popular than alternatives
  • May not match performance of larger or newer models

codebooga

Tags: coding

Owner/Author: oobabooga

Parameters: 34B

Resource Demand:High (34B)

Primary Use Cases: High-performing code instruct model, merged architecture

Pros:

  • High performance from merged model architecture
  • Specialized for instruction following
  • Large model size for complex tasks
  • Good for detailed coding instructions

Cons:

  • Very large model requires high-end hardware
  • Only one size option
  • Less popular than alternatives
  • Merged architecture may have compatibility considerations

starcoder

Tags: coding

Owner/Author: BigCode (Hugging Face)

Parameters: 1B, 3B, 7B, 15B

Resource Demand:Low (1B-7B), Medium (15B)

Primary Use Cases: Code generation across 80+ programming languages

Pros:

  • Trained on 80+ programming languages
  • Excellent multi-language support
  • Multiple size options
  • Well-established and reliable

Cons:

  • Older model (2 years) may lack latest improvements
  • May not specialize as well as newer models
  • Less popular than StarCoder2 successor

sqlcoder

Tags: coding

Owner/Author: Defog.ai

Parameters: 7B, 15B

Resource Demand:Low (7B), Medium (15B)

Primary Use Cases: SQL generation tasks, database query generation

Pros:

  • Specialized for SQL generation
  • Fine-tuned on StarCoder for SQL tasks
  • Two size options available
  • Excellent for database-related coding

Cons:

  • Specialized only for SQL, less versatile
  • May not perform well on non-SQL tasks
  • Limited use case compared to general models

wizardcoder

Tags: coding

Owner/Author: WizardLM Team

Parameters: 33B

Resource Demand:High (33B)

Primary Use Cases: State-of-the-art code generation

Pros:

  • State-of-the-art code generation capabilities
  • Large model size for complex tasks
  • Strong performance on code generation benchmarks
  • Well-regarded in coding community

Cons:

  • Very large model requires high-end hardware
  • Only one size option
  • Older model (2 years) may lack latest improvements

codeup

Tags: coding

Owner/Author: juyongjiang

Parameters: 13B

Resource Demand:Medium (13B)

Primary Use Cases: Code generation based on Llama2

Pros:

  • Based on proven Llama2 architecture
  • Good balance of size and performance
  • Reliable code generation capabilities
  • Well-tested foundation

Cons:

  • Only one size option
  • Older architecture (Llama2-based)
  • Less popular than newer alternatives
  • May not match latest model improvements

llama3.1

Tags: writing, coding

Owner/Author: Meta

Parameters: 8B, 70B, 405B

Resource Demand:Low (8B), High (70B), Very-High (405B)

Primary Use Cases: General-purpose tasks, creative writing, code generation, instruction following

Pros:

  • Extremely popular and well-tested (8.5M pulls)
  • Versatile for both writing and coding tasks
  • Excellent instruction following capabilities
  • Strong creative writing performance
  • Multiple size options including massive 405B model
  • Latest Llama architecture improvements

Cons:

  • 405B model requires extremely high-end hardware
  • May not specialize as well as dedicated models
  • General-purpose nature means less optimization for specific tasks

llama3

Tags: writing, coding

Owner/Author: Meta

Parameters: 8B, 70B

Resource Demand:Low (8B), High (70B)

Primary Use Cases: Creative writing, general-purpose tasks, code generation, storytelling

Pros:

  • Very popular creative writing model (6.2M pulls)
  • Excellent for storytelling and narrative content
  • Good understanding of nuance and tone
  • Versatile for both writing and coding
  • Well-established and reliable
  • Strong dialogue generation

Cons:

  • Older than Llama 3.1
  • 70B model requires significant resources
  • May not match specialized models in specific domains

llama3.2

Tags: writing, coding

Owner/Author: Meta

Parameters: 1B, 3B

Resource Demand:Low (1B-3B)

Primary Use Cases: Lightweight general-purpose tasks, writing, coding on resource-constrained devices

Pros:

  • Very lightweight models suitable for edge devices
  • Good performance relative to size
  • Versatile for both writing and coding
  • Latest Llama architecture in compact form
  • Fast inference on limited hardware

Cons:

  • Smaller models may lack depth for complex tasks
  • Limited to two size options
  • May not match larger models on complex reasoning

qwen2.5

Tags: writing, coding

Owner/Author: Alibaba Cloud (Qwen)

Parameters: 0.5B, 1.5B, 3B, 7B, 14B, 32B, 72B

Resource Demand:Low (0.5B-7B), Medium (14B), High (32B-72B)

Primary Use Cases: General-purpose tasks, creative writing, code generation, multilingual content

Pros:

  • Most popular general-purpose model (12.3M pulls)
  • Excellent for both writing and coding tasks
  • Wide range of sizes for different hardware
  • Strong multilingual capabilities
  • Versatile and well-tested
  • Good creative writing performance

Cons:

  • General-purpose nature means less specialization
  • Larger models (72B) require substantial resources
  • May not match dedicated models in specific domains

deepseek-r1

Tags: writing

Owner/Author: DeepSeek

Parameters: 1.5B, 7B, 14B, 32B, 70B

Resource Demand:Low (1.5B-7B), Medium (14B), High (32B-70B)

Primary Use Cases: Long-form writing, structured content generation, detailed reasoning

Pros:

  • Specialized for long-form content generation
  • Excellent structured writing capabilities
  • Strong reasoning and detailed outputs
  • Multiple size options
  • Well-optimized for writing tasks
  • Good for blog posts, articles, and essays

Cons:

  • Primarily focused on writing, less versatile for coding
  • Larger models require significant resources
  • Relatively new with less community testing

mistral

Tags: writing, coding

Owner/Author: Mistral AI

Parameters: 7B

Resource Demand:Low (7B)

Primary Use Cases: Creative writing, instruction following, general-purpose tasks, code generation

Pros:

  • Very popular and efficient (5.8M pulls)
  • Excellent instruction following
  • Good balance of writing and coding capabilities
  • Efficient 7B size suitable for most hardware
  • Well-tested and reliable
  • Strong creative writing performance

Cons:

  • Only one size option
  • May not match larger models on complex tasks
  • Older than newer Mistral variants

mixtral

Tags: writing, coding

Owner/Author: Mistral AI

Parameters: 8x7B (45B effective)

Resource Demand:High (8x7B)

Primary Use Cases: Complex creative writing, advanced code generation, mixture-of-experts efficiency

Pros:

  • Mixture-of-Experts architecture for efficiency
  • Excellent for complex creative tasks
  • Good performance on both writing and coding
  • More efficient than traditional 45B models
  • Strong for advanced use cases
  • Well-regarded in creative writing community

Cons:

  • Still requires substantial resources
  • Only one configuration available
  • May be overkill for simple tasks

gemma2

Tags: writing, coding

Owner/Author: Google

Parameters: 2B, 9B, 27B

Resource Demand:Low (2B), Medium (9B), Medium (27B)

Primary Use Cases: Narrative-driven content, creative writing, general-purpose tasks, code generation

Pros:

  • Google-developed with strong narrative capabilities
  • Good for upbeat, conversational writing
  • Versatile for both writing and coding
  • Multiple size options
  • Well-optimized for narrative content
  • Strong for brainstorming and creative tasks

Cons:

  • May not match specialized models in specific domains
  • Smaller models may lack depth
  • Less popular than some alternatives

gemma3

Tags: writing, coding

Owner/Author: Google

Parameters: 270M, 1B, 4B, 12B, 27B

Resource Demand:Low (270M-12B), Medium (27B)

Primary Use Cases: Lightweight writing tasks, narrative content, general-purpose, code generation

Pros:

  • Latest Gemma architecture
  • Very lightweight options (270M, 1B) for edge devices
  • Good for narrative and creative writing
  • Versatile for both writing and coding
  • Wide range of sizes
  • Fast inference on smaller models

Cons:

  • Relatively new with less community testing
  • Smaller models may lack depth
  • Less popular than Gemma2

dolphin-llama3

Tags: writing

Owner/Author: Eric Hartford (community)

Parameters: 8B, 70B

Resource Demand:Low (8B), High (70B)

Primary Use Cases: Uncensored creative writing, fiction, roleplay, immersive storytelling

Pros:

  • Uncensored variant for unrestricted creative writing
  • Excellent for fiction and immersive storytelling
  • Based on proven Llama3 architecture
  • Popular for roleplay scenarios
  • Two size options available
  • Strong narrative capabilities

Cons:

  • Uncensored nature may not be suitable for all use cases
  • Primarily focused on writing, less versatile
  • May have ethical considerations for some teams

dolphin-mistral

Tags: writing

Owner/Author: Eric Hartford (community)

Parameters: 7B

Resource Demand:Low (7B)

Primary Use Cases: Uncensored creative writing, fiction, roleplay, unrestricted content

Pros:

  • Uncensored variant based on Mistral
  • Efficient 7B size
  • Excellent for fiction and creative writing
  • Popular for unrestricted scenarios
  • Well-tested uncensored model

Cons:

  • Uncensored nature may not be suitable for all use cases
  • Only one size option
  • Primarily focused on writing

dolphin3

Tags: writing

Owner/Author: Eric Hartford (community)

Parameters: 8B

Resource Demand:Low (8B)

Primary Use Cases: Uncensored creative writing, fiction, roleplay, based on Llama 3.1

Pros:

  • Based on latest Llama 3.1 architecture
  • Uncensored for unrestricted creative writing
  • Good balance of size and performance
  • Popular for fiction and roleplay
  • Latest Dolphin variant

Cons:

  • Uncensored nature may not be suitable for all use cases
  • Only one size option
  • Primarily focused on writing

phi3

Tags: writing, coding

Owner/Author: Microsoft

Parameters: 3.8B, 14B

Resource Demand:Low (3.8B), Medium (14B)

Primary Use Cases: Lightweight writing tasks, structured content, code generation, edge devices

Pros:

  • Very lightweight and efficient
  • Good for structured writing and rubrics
  • Versatile for both writing and coding
  • Excellent for resource-constrained environments
  • Fast inference
  • Microsoft-developed with good documentation

Cons:

  • Smaller models may lack depth for complex tasks
  • Limited size options
  • May not match larger models on complex reasoning

vicuna

Tags: writing

Owner/Author: LMSYS

Parameters: 7B, 13B, 33B

Resource Demand:Low (7B), Medium (13B), High (33B)

Primary Use Cases: Natural conversational writing, custom assistants, dialogue generation

Pros:

  • Natural, less robotic conversational style
  • Excellent for dialogue and conversational content
  • Good for custom assistant applications
  • Multiple size options
  • Well-regarded for natural language generation

Cons:

  • Primarily focused on writing, less versatile
  • Older model may lack latest improvements
  • May not match newer models on complex tasks

ministral-3

Tags: writing, coding

Owner/Author: Mistral AI

Parameters: 3B, 8B, 14B

Resource Demand:Low (3B-8B), Medium (14B)

Primary Use Cases: Edge deployment, efficient writing and coding, resource-constrained environments

Pros:

  • Designed for edge deployment
  • Efficient models for limited resources
  • Versatile for both writing and coding
  • Good performance relative to size
  • Fast inference
  • Latest Mistral architecture in compact form

Cons:

  • Relatively new with less community testing
  • Smaller models may lack depth
  • Less popular than full-size Mistral

How to Choose the Right Open Source LLM

Selecting an open source LLM depends on hardware capabilities, use case, and performance requirements.

Hardware Constraints:

  • For limited resources: Consider smaller models like stable-code (3B), codegemma (2B/7B), qwen2.5-coder (0.5B-7B), phi3 (3.8B), or llama3.2 (1B/3B)
  • For high-end hardware: Larger models like qwen3-coder (480B), deepseek-coder-v2 (236B), llama3.1 (405B), or codellama (70B)

Use Case - Coding:

  • General coding: qwen2.5-coder, codellama, or deepseek-coder
  • SQL-specific: sqlcoder
  • Long context/agentic: qwen3-coder
  • Code completion: stable-code, codegeex4
  • Multi-language: starcoder or starcoder2
  • Versatile (coding + writing): qwen2.5, llama3.1, mistral, mixtral

Use Case - Writing:

  • Creative writing: llama3, llama3.1, mistral, gemma2
  • Long-form content: deepseek-r1
  • Fiction/roleplay: dolphin-llama3, dolphin-mistral, dolphin3
  • Conversational: vicuna
  • Lightweight writing: phi3, gemma3, llama3.2
  • Versatile (writing + coding): qwen2.5, llama3.1, mistral, mixtral

Popularity and Reliability:

  • Most tested: qwen2.5 (12.3M pulls), qwen2.5-coder (10.1M pulls), llama3.1 (8.5M pulls), llama3 (6.2M pulls), mistral (5.8M pulls)
  • Newest features: qwen3-coder (3 months), llama3.2 (recent), gemma3 (recent)

Benefits of Running Open Source LLMs Locally

Running open source language models locally has these characteristics compared to cloud-based APIs:

  • Privacy: Your code and conversations never leave your machine
  • Cost: No per-token API fees or subscription costs
  • Control: Full control over model versions, parameters, and data
  • Offline Access: Work without internet connectivity
  • Customization: Fine-tune models for your specific needs
  • No Rate Limits: Generate as much content as your hardware allows

Getting Started with Local LLMs

You can run open-source LLMs locally using several tools and platforms:

GUI-based tools:

  • LM Studio - Interface for downloading and chatting with models
  • Jan - Open-source ChatGPT alternative
  • GPT4All - General-purpose application with document chat capabilities

Command-line tools:

  • Ollama - Simple command-line tool for running models locally
  • llama.cpp - Lightweight C++ implementation that runs models efficiently on CPUs
  • Direct model loading via Python frameworks like Hugging Face Transformers, PyTorch, or TensorFlow

Web interfaces: If you want a ChatGPT-like experience, you can pair these backends with interfaces like LobeChat, Open WebUI, or LibreChat.

LM Studio or Jan provide model downloads and chat interfaces without command-line work. They support the same model formats (GGUF) that Ollama uses.

Code LLMs vs Writing Models: What’s the Difference?

Differences between code LLMs and writing models:

Code LLMs (like qwen2.5-coder, codellama, deepseek-coder) are trained on code repositories and handle:

  • Code generation and completion
  • Debugging and error fixing
  • Code explanation and documentation
  • Multi-language programming support
  • Understanding code context and syntax

Writing Models (like llama3.1, mistral, gemma2) are designed for natural language tasks:

  • Creative writing and storytelling
  • Content generation and editing
  • Conversational AI and chat
  • Long-form content creation
  • General language understanding

Versatile Models (like qwen2.5, llama3.1, mistral) handle both coding and writing tasks.

Using Ollama for Local LLM Deployment

Ollama provides a command-line interface and API for running open source LLMs locally. Example usage:

# Pull a model (coding example)
ollama pull qwen2.5-coder:7b

# Pull a model (writing example)
ollama pull llama3.1:8b

# Run a model
ollama run qwen2.5-coder:7b
ollama run llama3.1:8b

# Or use in your application (coding)
curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:7b",
  "prompt": "Write a Python function to calculate Fibonacci numbers."
}'

# Or use in your application (writing)
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Write a short story about a robot learning to paint."
}'

Open source language models are being used across various domains:

  • Code Generation: Automate boilerplate code, generate functions, and complete code snippets
  • Code Review: Analyze code for bugs, security issues, and best practices
  • Documentation: Generate API docs, README files, and technical documentation
  • Creative Writing: Draft stories, articles, and creative content
  • Content Editing: Improve grammar, style, and clarity of written content
  • Conversational AI: Build chatbots and virtual assistants
  • Data Analysis: Generate SQL queries and analyze datasets
  • Learning: Understand programming concepts and get coding help

Running Local LLMs

Considerations for running open source LLMs:

  • Start Small: Begin with smaller models (3B-7B parameters) to test your hardware
  • Monitor Resources: Use system monitoring tools to track GPU/CPU and memory usage
  • Experiment with Quantization: Use quantized models (Q4, Q5, Q8) to reduce memory requirements
  • Try Multiple Models: Different code LLMs and writing models perform differently on various tasks
  • Use Appropriate Context Windows: Match model context length to your use case
  • Keep Models Updated: Regularly pull updated versions for bug fixes and improvements

References and Resources