The Video-LLaMA project aims to empower large language models with video understanding capability. The project has introduced several language decoders and fine-tuned the video-language aligned models using machine-translated video chat instructions. The pre-trained and fine-tuned checkpoints can be downloaded from the provided links. The repository includes scripts to run demos and training on the Webvid-2.5M video caption dataset and LLaVA-CC3M image caption dataset. The project acknowledges several other projects that have contributed to the development of Video-LLaMA.

video_llama_logo.jpg

Video-LLaMA

ggml is a tensor library for machine learning to enable large models and high performance on commodity hardware. It is used by llama.cpp and whisper.cpp.

ggml.jpg

ggml

PrivateGPT is a software that allows users to ask questions to their documents without an internet connection, using the power of LLMs. The software is 100% private, and no data leaves the execution environment at any point. Users can ingest documents and ask questions without an internet connection. PrivateGPT is built with LangChain, GPT4All, LlamaCpp, Chroma, and SentenceTransformers. The software requires Python 3.10 or later and supports various file extensions, such as CSV, Word Document, EverNote, Email, EPub, PDF, PowerPoint Document, Text file (UTF-8), and more. Users can ingest multiple documents, and all will be accumulated in the local embeddings database. PrivateGPT uses a local LLM based on GPT4All-J or LlamaCpp to understand questions and create answers, and the context for the answers is extracted from the local vector store using a similarity search to locate the right piece of context from the docs. This is a test project, and it is not production-ready.

236942256-985801c9-25b9-48ef-80be-3acbb4575164.png

privateGPT

MosaicML allows you to easily train and deploy large AI models on your data, in your secure environment. With MosaicML, you can train and serve large AI models at scale with a single command and stay on the cutting edge with their latest recipes, techniques, and foundation models. You can deploy securely, run anywhere, own your model that's trained on your own data and seamlessly integrate with your existing data pipelines, experiment trackers, and other tools. Choose just the pieces you need from their modular training stack and modify their starter code however you want to implement your ideas. MosaicML has been trusted by ML Experts.

MosaicML.jpg

MosaicML

This utility allows you to test a variety of Large Language Models (LLMs) to compare and contrast their results. Large Language Models are massive neural networks that can generate human-like text through artificial intelligence to understand, generate, and manipulate human language. These models are typically based on deep learning techniques and trained on massive amounts of textual data to capture the nuances and complexity of human language. With this utility, you can discover and compare results from various LLMs simultaneously, including language translation, summarization, question-answering, text generation, and more. This Large Language Model Playground is completely free to use and is powered by Retool Embed, which allows you to turn internal tools and utilities like this LLM playground into public, customer-facing apps.

Large Language Models Playground.jpg

Large Language Models Playground

The StableLM repository contains a series of language models developed by Stability AI, trained on the new dataset, and continuously updated with new checkpoints. Currently, the StableLM-Alpha models are available, based on the Pile dataset, with 3B and 7B parameters, and will soon be followed by 15B and 30B models. StableLM-Tuned-Alpha, a fine-tuned model using a combination of five recent datasets for conversational agents, is available for chat. The models are hosted on Huggingface hub. A user can run inference using limited GPU capabilities with the help of a provided code snippet. The base model checkpoints are licensed under the Creative Commons license, while the fine-tuned checkpoints are licensed under the Non-Commercial Creative Commons license. The code in the repository is licensed under the Apache License 2.0 license.

mascot.png

StableLM

Run large language models like BLOOM-176B collaboratively — you load a small part of the model, then team up with people serving the other parts to run inference or fine-tuning. Single-batch inference runs at ≈ 1 sec per step (token) — up to 10x faster than offloading, enough for chatbots and other interactive apps. Parallel inference reaches hundreds of tokens/sec. Beyond classic language model APIs — you can employ any fine-tuning and sampling methods, execute custom paths through the model, or see its hidden states. You get the comforts of an API with the flexibility of PyTorch. Try now in Colab Docs on GitHub. Join our Discord or subscribe via email to follow Petals development. Featured on: This project is a part of the BigScience research workshop.

Petals.jpg

Petals

RedPajama is a project to create a reproducible, fully open, leading language model by reproducing LLaMA training dataset of over 1.2 trillion tokens. The project aims to remove limitations such as commercial APIs and provide a more transparent pipeline for research. RedPajama has three key components: pre-training data, base models, and instruction tuning data and models. The pre-training data is the first component that is already released, which includes a 1.2 trillion token fully-open dataset created by following the recipe described in the LLaMA paper. The project is a collaboration between Together, Ontocord.ai, ETH DS3Lab, Stanford CRFM, Hazy Research, and MILA Québec AI Institute.

RedPajama.jpg

RedPajama

MiniGPT-4 aligns a frozen visual encoder with a frozen large language model, Vicuna, using just one projection layer which possesses many capabilities similar to those exhibited by GPT-4 like detailed image description generation and website creation from hand-written drafts. Only performing the pretraining on raw image-text pairs produced unnatural language outputs that lack coherency including repetition and fragmented sentences. MiniGPT-4 consists of a vision encoder with a pretrained ViT and Q-Former, a single linear projection layer, and an advanced Vicuna large language model. MiniGPT-4 only requires training the linear layer to align the visual features with the Vicuna.

MiniGPT-4.jpg

MiniGPT-4

The JARVIS project is a collaborative system that connects numerous AI models for solving complicated AI tasks. The workflow of the system consists of four stages: task planning, model selection, task execution, and response generation. The project is under construction, but updates include CLI mode, Gradio demo, and built-in web API for tasks and results in server mode. The system's hardware requirements vary depending on the configuration, with a recommended minimum configuration of hybrid inference mode and minimal local deployment. The README file provides configuration details, installation instructions, and examples of use. The paper "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace" is available for reference, and the project is acknowledged by ChatGPT, Hugging Face, ControlNet, and ChatGPT-vue.

overview.jpg

JARVIS

The Pinecone vector database makes it easy to build high-performance vector search applications. Developer-friendly, fully managed, and easily scalable without infrastructure hassles.

Pinecone.jpg

Pinecone

This repository showcases how to turn your website content into a question answering bot or interactive chatbot using the LangChain tool and the OpenAI API. It includes three scripts for generating embeddings, asking questions, and starting a chat interface. The most important step is creating embeddings by looping over a sitemap.xml file and obtaining an OpenAI API key. Once created, the chatbot can provide answers with source document URLs. Dependencies can be installed with pip.

llm-qa-overview.png

content-chatbot

LangChain is an open source library that helps you build Large language models (LLMs) . There are six main areas that LangChain is designed to help with. These are, in increasing order of complexity:

📃 LLMs and Prompts:

This includes prompt management, prompt optimization, generic interface for all LLMs, and common utilities for working with LLMs.

🔗 Chains:

Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.

📚 Data Augmented Generation:

Data Augmented Generation involves specific types of chains that first interact with an external datasource to fetch data to use in the generation step. Examples of this include summarization of long pieces of text and question/answering over specific data sources.

🤖 Agents:

Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.

🧠 Memory:

Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.

lang.png

LangChain

llama.cpp is a plain C/C++ implementation without dependencies for inference of the LLaMA model. The main goal is to run the model using 4-bit quantization on a MacBook, with supported platforms including Mac OS, Linux, Windows, and Docker. The project supports different models and runs on CPU with mixed F16/F32 precision. The model is for educational purposes with new features that can be added mostly through community contributions. The README.md provides usage instructions for obtaining and verifying model data, running the model, and contributing guidelines.

227761327-6d83e30e-2200-41a6-bfbb-f575231c54f4.png

llama.cpp

We want to improve the world by providing amazing conversational AI. Open Assistant is a project organized by LAION and individuals around the world interested in bringing this technology to everyone. The code and models are licensed under the Apache 2.0 license and will be free to use and modify. Versions will be runnable on consumer hardware. All open source projects begin with people like you. Join us!

Open Assistant.jpg

Open Assistant

Open-source chatbots project for various applications. Provides an instruction-tuned large language model, customization recipes to fine-tune the model, an extensible retrieval system to augment the model with live-updating information, and a moderation model to filter inappropriate or out-of-domain questions.

OpenChatKit.jpg

OpenChatKit

Vicuna-13B is an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. It achieves more than 90%* quality of OpenAI ChatGPT and Google Bard while outperforming other models like LLaMA and Stanford Alpaca in more than 90%* of cases. The cost of training Vicuna-13B is around $300. The training and serving code, along with an online demo, are publicly available for non-commercial use.

Vicuna.jpg

Vicuna

Embed and store documents, images, and more. High-performance retrieval workflows with no additional work.

Baseplate.jpg

Baseplate

Vellum brings LLM-powered features to production with tools for prompt engineering, version control, back-testing, performance monitoring, and continuous fine-tuning. Compatible across all major LLM providers.

Browse 19+ Llm tools