Create README.md

README.md

@@ -0,0 +1,51 @@
+---
+license: mit
+base_model:
+- unsloth/DeepSeek-R1-BF16
+---
+## Model Details
+
+This model card is for mxfp4 quantization of [unsloth/DeepSeek-R1-BF16](https://huggingface.co/unsloth/DeepSeek-R1-BF16) based on [intel/auto-round](https://github.com/intel/auto-round) saved in llm_compressor format. 
+Please follow the license of the original model.
+
+## How to Use
+
+The step-by-step README of quantization and evaluation can be found in [Intel Neural Compressor Examples](https://github.com/intel/neural-compressor/blob/master/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/deepseek/README.md).
+
+## Evaluate
+
+```
+# auto-round 0.14.2
+# vllm 0.26.1rc1.dev451+gb1e12d142
+
+export CUDA_VISIBLE_DEVICES=4,5,6,7
+export VLLM_QDQ=1
+python vllm/examples/basic/offline_inference/generate.py \
+        --model /software/data/jenkins/saved_models/DeepSeek-R1_mxfp4_LLMC/DeepSeek-R1-BF16-mxfp-w4g32/ \
+        --max-model-len 2048 \
+        -tp 4 \
+        --enforce-eager \
+        --gpu-memory-utilization 0.9
+```
+
+## Ethical Considerations and Limitations
+
+The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. 
+Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
+
+Therefore, before deploying any applications of the model, developers should perform safety testing.
+
+## Caveats and Recommendations
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
+
+Here are a couple of useful links to learn more about Intel's AI software:
+
+- [Intel Neural Compressor](https://github.com/intel/neural-compressor)
+- [AutoRound](https://github.com/intel/auto-round)
+
+## Disclaimer
+
+The license on this model does not constitute legal advice. 
+We are not responsible for the actions of third parties who use this model. 
+Please consult an attorney before using this model for commercial purposes.