Hosting Your Own LLM: 100 times Cheaper
The main benefit of hosting your own AI is that you own the Intellectual Property of all conversations.
The second benefit is cost
Most businesses that need AI start with a cloud solution. The most popular are OpenAI’s chatGPT, Azure’s turbo GPT, and AWS-hosted huggingface models, and the benefit is a marginal set-up cost. However, when operations ramp-up as the AI traffic grows, these solutions cost thousands of dollars per month. The solution is owning your AI.
Cheaper than the cloud
If you need your chatbot available 24/7, AWS hosting gets very expensive very quickly.
| Platform | Cost per Hour | Cost per Day | Cost per Month | Cost per Year |
|---|---|---|---|---|
| L&K Tech LLM in a box | $0.01 | $0.3 | $8.6 | $100 |
| EC2 AWS (g4dn.12xlarge) | $2-4 | $48 | $1,440 | $17,280 |
While the cost of L&K Tech LLM in a box per hour is only the electrity used, the upfront cost of $10,000 eliminates the need for ongoing hourly charges. In comparison, an EC2 AWS instance of equal power can cost between $2-4 per hour, resulting in a significant expense over time. Running the same models in the cloud costs 170 times more!
Cheaper than GPT
| Platform | Tokens per Second | Tokens per Day | Cost per 1000 Tokens |
|---|---|---|---|
| L&K Tech LLM in a box | 16 | 1,382,400 | $0.0002 |
| OpenAI GPT 3.5 | 15 | 1,296,000 | $0.002 |
| OpenAI GPT 4 | 4 | 345,600 | $0.06 |
L&K Tech LLM in a box can generate 30 tokens per second, resulting in 43,000 tokens per day. With a cost of $0.015 per 1000 tokens, it offers an affordable solution for token generation. In contrast, OpenAI GPT 4 costs $0.06 per 1000 tokens, making it significantly more expensive. Our AI solution is 300 times cheaper than GPT 4!
Cheaper than self-built
State-of-the-art models require expensive specialised hardware, includign multiple GPUs or TPUs. L&K Tech has optimised for this: our hardware is the least expensive solution that suits your needs. Thanks to our unique relationship with NVIDIA, we are able to provide the Teraflops needed at a fraction of the cost from building your own hardware. You will spend between 12 and 40 thousand dollars on components to run a 33B parameter model.
Furthermore, businesses typically spend 4 to 6 months building and deploying their custom API. Consider the staff costs involved and L&K Tech AI in a box is a clear net benefit.
| Platform | upfront cost |
|---|---|
| L&K Tech LLM in a box | $10,000 |
| Own GPUs | $8,000 to $30,000 |
| Own server | $4,000 to $10,000 |
| Own staff costs | $20,000 to $100,000 |
| Own total | $12,000 to $40,000 plus staff |
Accuracy Comparison
While GPT-4 has shown a decline in accuracy over time, L&K-NLP, powered by the newest models on the HuggingFace Leaderboard, continues to improve. According to the HuggingFace Leaderboard (https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard), L&K-NLP achieves an accuracy rating of 87.6, surpassing GPT-3.5’s rating of 79.4. Furthermore, GPT-4, initially released with an accuracy rating of 95.4, has now dropped to 85.2, highlighting the need for consistent performance of L&K Tech’s LLM.
Conclusion
Hosting your own LLM can provide a cost-effective, efficient, and accurate solution for businesses. With lower costs compared to AWS, affordable token generation, and superior accuracy, L&K Tech LLM in a box offers a compelling alternative to relying on third-party services. Stay ahead of the competition and harness the power of L&K-NLP for your business needs.