AnswerToSpeech

Use this node in question-answering pipelines to convert text Answers into SpeechAnswers. The answer and its context are read out into two audio files.

AnswerToSpeech lives in the haystack-extras repo, it's not part of Haystack core. This node is experimental because of the data classes it uses (SpeechAnswer). Bear in mind that they might change in the future.

Position in a PipelineThe last node in the pipeline, after a Reader
InputAnswer
OutputSpeechAnswer
ClassesAnswerToSpeech

Installation

AnswerToSpeech is not installed as part of Haystack core, you need to install it separately:

# First, install the audio system dependencies:
sudo apt-get install libsndfile1 ffmpeg

# Now, install the node:
pip install farm-haystack-text2speech

Usage

To initialize AnswerToSpeech, run:

from text2speech import AnswerToSpeech

model_name = 'espnet/kan-bayashi_ljspeech_vits'
answer_dir = './generated_audio_answers'

audio_answer = AnswerToSpeech(model_name_or_path=model_name, generated_audio_dir=answer_dir)

To use AnswerToSpeech in a pipeline, run:

from text2speech import AnswerToSpeech

retriever = BM25Retriever(document_store=document_store)
reader = FARMReader(model_name_or_path="deepset/roberta-base-squad2", use_gpu=True)
answer2speech = AnswerToSpeech(
    model_name_or_path="espnet/kan-bayashi_ljspeech_vits",
    generated_audio_dir=Path(__file__).parent / "audio_answers",
    )

audio_pipeline = Pipeline()
audio_pipeline.add_node(retriever, name="Retriever", inputs=["Query"])
audio_pipeline.add_node(reader, name="Reader", inputs=["Retriever"])
audio_pipeline.add_node(answer2speech, name="AnswerToSpeech", inputs=["Reader"])