Instructions to use preemware/Prox-1.0-GPT-6.7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use preemware/Prox-1.0-GPT-6.7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="preemware/Prox-1.0-GPT-6.7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("preemware/Prox-1.0-GPT-6.7B")
model = AutoModelForCausalLM.from_pretrained("preemware/Prox-1.0-GPT-6.7B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use preemware/Prox-1.0-GPT-6.7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "preemware/Prox-1.0-GPT-6.7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "preemware/Prox-1.0-GPT-6.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/preemware/Prox-1.0-GPT-6.7B

SGLang

How to use preemware/Prox-1.0-GPT-6.7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "preemware/Prox-1.0-GPT-6.7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "preemware/Prox-1.0-GPT-6.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "preemware/Prox-1.0-GPT-6.7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "preemware/Prox-1.0-GPT-6.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use preemware/Prox-1.0-GPT-6.7B with Docker Model Runner:
```
docker model run hf.co/preemware/Prox-1.0-GPT-6.7B
```

Anthony commited on Jan 15, 2024

Commit

5f04c8b

verified ·

1 Parent(s): e003e97

Update README.md

Browse files

Files changed (1) hide show

README.md +15 -13

README.md CHANGED Viewed

@@ -1,4 +1,5 @@
 ---
 language: en
 license: apache-2.0
 tags:
@@ -7,23 +8,24 @@ tags:
 - Penetration Testing
 - Uncensored
 - Synthetic Data
-- GPT-2.7B
 model-index:
-  - name: prox-1.0-gpt-6.7b
-    results: []
 ---
-# Prox-1.0-GPT-6.7B by OpenVoid
 ![Prox Model Logo](https://data.openvoid.ai/images/prox.png)
-*Drawing inspiration from the concept of 'proximity' in digital networks, the Prox series by OpenVoid stands at the forefront of cybersecurity technology. Prox-1.0-GPT-6.7B embodies this ethos, offering cutting-edge solutions in the realm of cyber security and penetration testing.*
 ## Model Description
-Prox-1.0-GPT-6.7B is an advanced language model tailored for cybersecurity applications. It excels in red teaming tasks, including the analysis and simulation of phishing emails. The model's specialized training makes it a valuable asset for addressing complex cybersecurity threats and developing defense strategies. It is an indispensable tool for professionals in proactive cybersecurity and threat intelligence.
 ## Training
-Prox-1.0-GPT-6.7B was trained over 12 hours for 4 epochs on 8x A100 GPUs using Qlora and Axolotl.
 Prompt format:
 This model uses ChatML prompt format.
@@ -45,8 +47,8 @@ What is SQL Injection?<|im_end|>
 <|im_start|>assistant
 ```
-## Community and Support
-- **Stay Updated**: Follow OpenVoid's updates via [Twitter](https://twitter.com/OpenVoidAI).
-[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)

 ---
+base_model: cerebras/Cerebras-GPT-6.7B
 language: en
 license: apache-2.0
 tags:
 - Penetration Testing
 - Uncensored
 - Synthetic Data
 model-index:
+- name: Prox-1.0-GPT-6.7B
+  results: []
+datasets:
+- teknium/openhermes
+- cognitivecomputations/dolphin
+- LDJnr/Capybara
+- garage-bAInd/Open-Platypus
 ---
+# Prox-1.0-GPT-6.7B
 ![Prox Model Logo](https://data.openvoid.ai/images/prox.png)
 ## Model Description
+Prox-1.0-GPT-6.7B is a fine-tuned version of Cerebras-GPT-6.7B, specifically tailored for cybersecurity. It has been trained on the Prox V1 dataset and incorporates high-quality open datasets from various sources within the AI community. Additionally, it has been equipped with basic chat abilities through training on the Dolphin and OpenHermes datasets.
 ## Training
+Prox-1.0-GPT-6.7B was trained over 12 hours for 4 epochs on 8x A100 GPUs using Qlora.
 Prompt format:
 This model uses ChatML prompt format.
 <|im_start|>assistant
 ```
+## Misuse, Malicious Use, and Out-of-Scope Use
+Users are responsible for their applications of this model. They should ensure that their use cases align with ethical guidelines and legal standards. Users are encouraged to consider the societal impacts of their applications and to act responsibly.
+## License
+The weights of Prox-1.0-GPT-6.7B are licensed under version 2.0 of the Apache License.