# How to use the inference api on tts model?

**URL:** <https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397>\
**Category:** Models\
**Created:** [November 30, 2021, 3:04pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397 "2021-11-30T15:04:08Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![danijelpetkovic](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@danijelpetkovic](https://discuss.huggingface.co/u/danijelpetkovic)\
**Post date:** [November 30, 2021, 3:04pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/1 "2021-11-30T15:04:08Z")

</div>

Hi, how can I use the inference api on this model: espnet/kan-bayashi\_ljspeech\_vits?  
It receives text and should return audio file, since it is text to speech model.

---

<div class="post-metadata">

**Author:** ![osanseviero](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/osanseviero/32/38738_2.png) [@osanseviero](https://discuss.huggingface.co/u/osanseviero)\
**Post date:** [December 1, 2021, 8:48am UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/2 "2021-12-01T08:48:41Z")

</div>

Hey there. Here is a snippet to do this. It will return the raw data, so you will still need to use something such as `ffmpeg` to process it. cc @Narsil

```auto
import json
import requests

headers = {"Authorization": f"Bearer {API_TOKEN}"}
API_URL = "https://api-inference.huggingface.co/models/espnet/kan-bayashi_ljspeech_vits"

def query(payload):
    data = json.dumps(payload)
    response = requests.request("POST", API_URL, headers=headers, data=data)
    return response.content

data = query({"inputs": "test"})

```

---

<div class="post-metadata">

**Author:** ![danijelpetkovic](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@danijelpetkovic](https://discuss.huggingface.co/u/danijelpetkovic)\
**Post date:** [December 1, 2021, 3:43pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/3 "2021-12-01T15:43:37Z")

</div>

@osanseviero Hey! Thanks for the help. I managed to call the api and to get a response:)  
Regarding processing of the raw data, does it mean I should run it throw ffmpeg and create an audio file from it, or you meant something different? Thanks again!

---

<div class="post-metadata">

**Author:** ![osanseviero](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/osanseviero/32/38738_2.png) [@osanseviero](https://discuss.huggingface.co/u/osanseviero)\
**Post date:** [December 1, 2021, 3:57pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/4 "2021-12-01T15:57:31Z")

</div>

@Narsil might know what’s the most recommended way to load the audio in this point. I usually use [ffmpeg](https://github.com/huggingface/huggingface_hub/blob/main/api-inference-community/api_inference_community/validation.py#L297) but in the context of small tests and quick debugging. I’ll let Nicolas give a more proper answer.

Btw, we have an audio study group forming in our Discord server: [Hugging Face](http://hf.co/join/discord). The first session will be in two weeks in case you’re interested.

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [December 3, 2021, 9:54am UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/5 "2021-12-03T09:54:18Z")

</div>

@danijelpetkovic ,

At this point you should have received a real audio file. If you just want to listen you can just save it

```python
with open("out.flac", "wb") as f:
    f.write(response.content)

```

And in order to listen to it, use your favorite media player.

`vlc out.flac` for me. Windows media player, winamp, foobar, chrome and firefox can probably read it too.

`ffmpeg` is necessary only if want multiple files and want to compress them a little. The API responds flac files, which are lossless compression, but you can get much better compression with lossy algorithm (all to save disk, it might not be necessary to you)

---

<div class="post-metadata">

**Author:** ![danijelpetkovic](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@danijelpetkovic](https://discuss.huggingface.co/u/danijelpetkovic)\
**Post date:** [December 14, 2021, 6:36pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/6 "2021-12-14T18:36:16Z")

</div>

@Narsil Thank you very much! I managed to save and play the file exactly as you described above. The only thing that I still can’t figure out is why the audio is distorted, it is not the same voice as it was when I was calling the model directly. I tried to add parameters to the inference api:

```auto
audio_file = query_audio_tts({
            "inputs": generated_answer,
            "parameters": {
                "vocoder_tag": "str_or_none(none)",
                "threshold": 0.5,
                "minlenratio": 0.0,
                "maxlenratio": 10.0,
                "use_att_constraint": False,
                "backward_window": 1,
                "forward_window": 3,
                "speed_control_alpha": 1.0,
                "noise_scale": 0.333,
                "noise_scale_dur": 0.333
            }
        })

```

But this didn’t help. Do you have any ideas here? Thanks!

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [December 14, 2021, 8:07pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/7 "2021-12-14T20:07:06Z")

</div>

@danijelpetkovic Do you have a reference for how you create your files without distortion ?

Currently `espnet` and other libraries (except transformers) don’t support adding parameters unfortunately. Since every library will have a different set of parameters, and maintaining each and everyone of them will very fast become tedious and probably clashes and bugs (not even mentioning the docs for that).

That being said, if we can improve the defaults for some models we should definitely do so, the implementations for this API is defined here: [huggingface\_hub/automatic\_speech\_recognition.py at main · huggingface/huggingface\_hub · GitHub](https://github.com/huggingface/huggingface_hub/blob/main/api-inference-community/docker_images/espnet/app/pipelines/automatic_speech_recognition.py)

As you can see it’s pretty bare bones, any help to improve that is welcome.

---

<div class="post-metadata">

**Author:** ![danijelpetkovic](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@danijelpetkovic](https://discuss.huggingface.co/u/danijelpetkovic)\
**Post date:** [December 15, 2021, 9:41am UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/8 "2021-12-15T09:41:41Z")

</div>

@Narsil Yes, sure. Here is the code chunk which I was using before:

```auto
from espnet2.bin.tts_inference import Text2Speech
from espnet2.utils.types import str_or_none
import torch
import scipy.io.wavfile

tag = 'kan-bayashi/ljspeech_vits' 
vocoder_tag = "none"

text2speech = Text2Speech.from_pretrained(
    model_tag=str_or_none(tag),
    vocoder_tag=str_or_none(vocoder_tag),
    device="cpu",
    # Only for Tacotron 2 & Transformer
    threshold=0.5,
    # Only for Tacotron 2
    minlenratio=0.0,
    maxlenratio=10.0,
    use_att_constraint=False,
    backward_window=1,
    forward_window=3,
    # Only for FastSpeech & FastSpeech2 & VITS
    speed_control_alpha=1.0,
    # Only for VITS
    noise_scale=0.333,
    noise_scale_dur=0.333,
)

def get_audio_tts(text):
  with torch.no_grad():
    wav = text2speech(text)["wav"]
    scipy.io.wavfile.write("out.wav",text2speech.fs , wav.view(-1).cpu().numpy())
  return "out.wav"

audio_file = get_audio_tts(generated_answer)

```

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [December 15, 2021, 9:58am UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/9 "2021-12-15T09:58:41Z")

</div>

Those options do seem to depend on the model looking at the comments.

Do you think you could maybe create a PR here: [huggingface\_hub/automatic\_speech\_recognition.py at main · huggingface/huggingface\_hub · GitHub](https://github.com/huggingface/huggingface_hub/blob/main/api-inference-community/docker_images/espnet/app/pipelines/automatic_speech_recognition.py) ?

We could ping some `espnet` maintainers to take a look.

---

<div class="post-metadata">

**Author:** ![danijelpetkovic](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@danijelpetkovic](https://discuss.huggingface.co/u/danijelpetkovic)\
**Post date:** [December 15, 2021, 3:47pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/10 "2021-12-15T15:47:31Z")

</div>

@Narsil I created a new issue and referenced the file you sent me.

> <https://github.com/huggingface/huggingface_hub/issues/540>
>
> https://github.com/huggingface/huggingface\_hub/blob/eeeb0d1b352fb3249541bb62f1f5…7f41ae3ab4e0/api-inference-community/docker\_images/espnet/app/pipelines/automatic\_speech\_recognition.py#L8
> 
> Hey guys. I tried to use the mentioned model via inference api and the audio that comes as an response is distorted.
> 
> When I try to import the model and call it directly it returns a nice audio with nice voice:
> 
> \`\`\`
> from espnet2.bin.tts\_inference import Text2Speech
> from espnet2.utils.types import str\_or\_none
> 
> tag = 'kan-bayashi/ljspeech\_vits'
> vocoder\_tag = "none"
> 
> text2speech = Text2Speech.from\_pretrained(
> model\_tag=str\_or\_none(tag),
> vocoder\_tag=str\_or\_none(vocoder\_tag),
> device="cpu",
> threshold=0.5,
> minlenratio=0.0,
> maxlenratio=10.0,
> use\_att\_constraint=False,
> backward\_window=1,
> forward\_window=3,
> speed\_control\_alpha=1.0,
> noise\_scale=0.333,
> noise\_scale\_dur=0.333,
> )
> 
> \`\`\`

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [December 16, 2021, 9:47am UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/11 "2021-12-16T09:47:37Z")

</div>

Turns out the issue lied with the sampling rate in fact. Here is the fix [Fixing FS for `espnet`. by Narsil · Pull Request #542 · huggingface/huggingface\_hub · GitHub](https://github.com/huggingface/huggingface_hub/pull/542)

---

<div class="post-metadata">

**Author:** ![danijelpetkovic](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@danijelpetkovic](https://discuss.huggingface.co/u/danijelpetkovic)\
**Post date:** [December 16, 2021, 10:03am UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/12 "2021-12-16T10:03:08Z")

</div>

@Narsil Thanks for a fast response and the fix. I will test it in the app and let you know!

---

<div class="post-metadata">

**Author:** ![danijelpetkovic](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@danijelpetkovic](https://discuss.huggingface.co/u/danijelpetkovic)\
**Post date:** [December 20, 2021, 1:01pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/13 "2021-12-20T13:01:06Z")

</div>

@Narsil Hey. Sorry if I am missing something, maybe I misunderstood. But I tried the response of the api after the fix from above and it still keeps returning distorted voice.

---

<div class="post-metadata">

**Author:** ![danijelpetkovic](https://avatars.discourse-cdn.com/v4/letter/d/e8c25b/32.png) [@danijelpetkovic](https://discuss.huggingface.co/u/danijelpetkovic)\
**Post date:** [December 23, 2021, 5:41pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/14 "2021-12-23T17:41:13Z")

</div>

@Narsil Hey 🙂 I found an interesting thing and I created a small app to show the problem. Depending on the content sent to the tts model, the voice is being returned differently.

> **[Test Tts Inference Api - a Hugging Face Space by danijelpetkovic](https://huggingface.co/spaces/danijelpetkovic/test-tts-inference-api)**
>
> Discover amazing ML apps made by the community

You can just try to paste this 2 slightly different paragraphs and you will see the difference in the voice.

“Water heated to room temperature feels colder than the air around it. This is because the temperature difference between the water and the air is greater than that of the air surrounding it.”

“Water heated to room temperature feels colder than the air around it. This is because the temperature difference between water and air is greater than the difference between the temperature of the water and the air.”

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [January 3, 2022, 1:58pm UTC](https://discuss.huggingface.co/t/how-to-use-the-inference-api-on-tts-model/12397/15 "2022-01-03T13:58:01Z")

</div>

This looks like a cache issue (There’s a cache in front of the API to prevent calculating things over and over).

You can try adding `{"inputs": "....", {"parameters": {"use_cache": False}}` to your input to force the output to be calculated.

The caching mechanism should be upgraded at some point so you don’t have to do this.