A self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.
umbrel.com ยป
## Demo
https://github.com/getumbrel/llama-gpt/assets/10330103/5d1a76b8-ed03-4a51-90bd-12ebfaf1e6cd
## How to install
### Install LlamaGPT on your umbrelOS home server
Running LlamaGPT on an [umbrelOS](https://umbrel.com) home server is one click. Simply install it from the [Umbrel App Store](https://apps.umbrel.com/app/llama-gpt).
[](https://apps.umbrel.com/app/llama-gpt)
### Install LlamaGPT anywhere else
You can run LlamaGPT on any x86 or arm64 system. Make sure you have Docker installed.
Then, clone this repo and `cd` into it:
```
git clone https://github.com/getumbrel/llama-gpt.git
cd llama-gpt
```
You can now run LlamaGPT with any of the following models depending upon your hardware:
| Model size | Model used | Minimum RAM required | How to start LlamaGPT |
| ---------- | ----------------------------------- | -------------------- | ------------------------------------------------ |
| 7B | Nous Hermes Llama 2 7B (GGML q4_0) | 8GB | `docker compose up -d` |
| 13B | Nous Hermes Llama 2 13B (GGML q4_0) | 16GB | `docker compose -f docker-compose-13b.yml up -d` |
| 70B | Meta Llama 2 70B Chat (GGML q4_0) | 48GB | `docker compose -f docker-compose-70b.yml up -d` |
You can access LlamaGPT at `http://localhost:3000`.
To stop LlamaGPT, run:
```
docker compose down
```
## Acknowledgements
A massive thank you to the following developers and teams for making LlamaGPT possible:
- [Mckay Wrigley](https://github.com/mckaywrigley) for building [Chatbot UI](https://github.com/mckaywrigley).
- [Georgi Gerganov](https://github.com/ggerganov) for implementing [llama.cpp](https://github.com/ggerganov/llama.cpp).
- [Andrei](https://github.com/abetlen) for building the [Python bindings for llama.cpp](https://github.com/abetlen/llama-cpp-python).
- [NousResearch](https://nousresearch.com) for [fine-tuning the Llama 2 7B and 13B models](https://huggingface.co/NousResearch).
- [Tom Jobbins](https://huggingface.co/TheBloke) for [quantizing the Llama 2 models](https://huggingface.co/TheBloke/Nous-Hermes-Llama-2-7B-GGML).
- [Meta](https://ai.meta.com/llama) for releasing Llama 2 under a permissive license.
---
[](https://github.com/getumbrel/llama-gpt/blob/master/LICENSE.md)
[umbrel.com](https://umbrel.com)
