【JAVA】使用vosk和FunSAR、whisper-cli实现windows实时语音转文字

【JAVA】使用vosk和FunSAR、whisper-cli实现windows实时语音转文字解放双手使用VOSK使用FUNASR参考模型下载安装包安装windows ffmpeg安装代码使用whisper.cpp安装依赖cpp下载地址下载模型如何使用python代码调用总结使用VOSK使用到的三方包jna-5.11.0.jar jna-platform-5.11.0.jar vosk-0.3.45.jar具体代码实现packagesample;importorg.vosk.Model;importorg.vosk.Recognizer;importjavax.sound.sampled.*;importjava.awt.*;importjava.awt.datatransfer.StringSelection;importjava.awt.event.KeyEvent;importjava.io.File;importjava.io.FileInputStream;importjava.io.IOException;/** * https://alphacephei.com/vosk/models * vosk.alphacephei.com * https://repo1.maven.org/maven2/com/alphacephei/vosk/0.3.45/ */publicclassVoskJavaExample{// 音频格式参数配置privatestaticfinalAudioFormat.EncodingENCODINGAudioFormat.Encoding.PCM_SIGNED;privatestaticfinalfloatRATE16000.0f;// 采样率privatestaticfinalintSAMPLE_SIZE16;// 采样位数privatestaticfinalintCHANNELS1;// 声道数1-单声道2-立体声privatestaticfinalbooleanBIG_ENDIANtrue;// 字节顺序publicstaticvoidmain(String[]args){writeOutResult(getVoiceDoc());}publicstaticvoidwriteOutResult(StringwavPath){try{// 1. 加载模型替换为你的模型路径如vosk-model-cn-0.22// Model model new Model(D:\\download\\vosk-model-cn-0.22\\vosk-model-cn-0.22);ModelmodelnewModel(D:\\download\\vosk-model-small-cn-0.22\\vosk-model-small-cn-0.22);// 2. 初始化识别器采样率必须与音频文件一致此处为16000Hzorg.vosk.Recognizerrecognizerneworg.vosk.Recognizer(model,16000.0f);// 3. 读取音频文件需为符合要求的WAV文件try(FileInputStreamfisnewFileInputStream(wavPath)){byte[]buffernewbyte[4096];// 缓冲区大小intbytesRead;// 4. 逐段处理音频流while((bytesReadfis.read(buffer))!-1){// 调用acceptWaveform传入字节数组和实际读取的长度if(recognizer.acceptWaveForm(buffer,bytesRead)){// 输出实时识别结果Stringresultrecognizer.getResult();System.out.println(实时结果result);}}// 5. 处理剩余音频输出最终结果Stringresultrecognizer.getFinalResult();System.out.println(最终结果result);resultresult.replaceAll(},).replaceAll(\text\ : \,).replaceAll(\,).replaceAll(\\{,).replaceAll( ,);System.out.println(最终结果1result);typeChinese(result);}// 释放资源model.close();recognizer.close();}catch(IOExceptione){e.printStackTrace();}}publicstaticStringgetVoiceDoc(){try{// 1. 设置音频格式AudioFormatformatnewAudioFormat(ENCODING,RATE,SAMPLE_SIZE,CHANNELS,(SAMPLE_SIZE/8)*CHANNELS,RATE,BIG_ENDIAN);// 2. 获取音频输入设备麦克风DataLine.InfoinfonewDataLine.Info(TargetDataLine.class,format);if(!AudioSystem.isLineSupported(info)){System.err.println(不支持指定的音频格式);returnnull;}// 3. 打开并启动录音线路TargetDataLineline(TargetDataLine)AudioSystem.getLine(info);line.open(format);line.start();System.out.println(开始录音...按回车键停止);// 4. 创建音频输入流AudioInputStreamaudioInputStreamnewAudioInputStream(line);// 5. 创建输出文件FileaudioFilenewFile(D:\\download\\savedVoice\\recording.wav);// 6. 启动独立线程处理录音停止newThread(()-{try{// 等待用户输入停止System.in.read();line.stop();line.close();System.out.println(录音已停止);}catch(IOExceptione){e.printStackTrace();}}).start();// 7. 将音频流写入文件AudioSystem.write(audioInputStream,AudioFileFormat.Type.WAVE,audioFile);System.out.println(录音已保存到audioFile.getAbsolutePath());}catch(LineUnavailableException|IOExceptione){e.printStackTrace();}returnD:\\download\\savedVoice\\recording.wav;}/** * 模拟输入中文文本通过剪贴板粘贴实现 * param text 要输入的中文文本 */privatestaticvoidtypeChinese(Stringtext){Robotrobotnull;try{robotnewRobot();}catch(AWTExceptione){e.printStackTrace();}// 1. 将文本复制到系统剪贴板StringSelectionselectionnewStringSelection(text);Toolkit.getDefaultToolkit().getSystemClipboard().setContents(selection,null);// 2. 模拟CtrlV粘贴robot.keyPress(KeyEvent.VK_CONTROL);// 按下Ctrlrobot.keyPress(KeyEvent.VK_V);// 按下Vrobot.delay(100);// 短暂延迟确保粘贴生效robot.keyRelease(KeyEvent.VK_V);// 释放Vrobot.keyRelease(KeyEvent.VK_CONTROL);// 释放Ctrl// 3. 延迟一段时间避免连续操作冲突robot.delay(300);}}代码最新版本https://gitee.com/likexiang/source-code-records/blob/master/java/VoiceToTxt.java使用FUNASR参考github链接https://github.com/modelscope/FunASR/blob/main/README_zh.md模型下载punc使用的模型punc_model的参数【加标点】punc_ct-transformer_zh-cn-common-vocab272727-pytorchhttps://www.modelscope.cn/models/iic/punc_ct-transformer_cn-en-common-vocab471067-large/files主要使用的模型model的参数【Paraformer语音识别-中文-通用-16k-离线-large-长音频版】speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorchhttps://www.modelscope.cn/models/iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch安装包安装python3.8torch1.13torchaudiopip3 install -U funasrpip3 install -U torchaudiopip3 install -U torchpip3 install -U ffmpegpip3 install -U bottlewindows ffmpeg安装选择版本678ffmpeg下载地址https://github.com/BtbN/FFmpeg-Builds/releases代码起一个简易的http服务器用于开放语音识别的借口这写死了recording.wav的具体位置然后使用java代码先把语音文件传过来然后java再使用httpget获取识别结果。fromfunasrimportAutoModelfrombottleimportroute,run,getfromdatetimeimportdatetime nowdatetime.now()date_time_strnow.strftime(%Y年%m月%d日 %H时%M分%S秒)print(开始加载,date_time_str)modelAutoModel(disable_updateTrue,modelD:\\voiceRecogize\\pythonVR\\speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404,punc_modelD:\\voiceRecogize\\pythonVR\\punc_ct-transformer_zh-cn-common-vocab272727-pytorch)# model AutoModel(modelD:\\voiceRecogize\\pythonVR\\speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch,punc_modelD:\\voiceRecogize\\pythonVR\\punc_ct-transformer_zh-cn-common-vocab272727-pytorch)modelPuncAutoModel(modelD:\\voiceRecogize\\pythonVR\\punc_ct-transformer_zh-cn-common-vocab272727-pytorch)nowdatetime.now()date_time_strnow.strftime(%Y年%m月%d日 %H时%M分%S秒)print(加载完成,date_time_str)# route(/)# def index():# return 你说说看先!route(/:content)defback(content):resmodel.generate(inputD:\\download\\serverFile\\recording.wav,hotword关联单)# [{key: recording, text: 我 在 群 里 讲 一 下, timestamp: [[390, 570], [570, 730], [730, 850], [850, 1010], [1010, 1250], [1250, 1350], [1350, 1765]]}]# print(res)# return 回答记录如下brbr%s % (res[0][text])finalResultres[0][text].replace( ,)remodelPunc.generate(inputfinalResult)print(re)returnre[0][text]run(host0.0.0.0,port8811)完整代码见https://gitee.com/likexiang/source-code-records/tree/master/java/VoiceServerWithNettyAndJavafx使用whisper.cpp安装依赖pip install pywhispercpp pyaudio pyautoguicpp下载地址这个release有两种一种发布的源代码一种发布的是bin文件我们这里下载bin文件如https://github.com/ggml-org/whisper.cpp/releases#release-b5130下载模型https://hf-mirror.com/ggerganov/whisper.cpp/tree/main从里面下载如下模型模型文件名有ggml-base.binggml-medium.binggml-large-v3.binggml-large-v3-turbo.bin如何使用./whisper-cli -m ggml-large-v3-turbo.bin -f 11.wav -l zh --output-txtpython代码调用frompywhispercpp.modelimportModelimportpyaudioimportwaveimportthreadingimporttimeimportpyautogui# 配置区 MODEL_PATHr./ggml-large-v3-turbo.bin# ggml模型路径LANGUAGEzhSAMPLE_RATE16000# whisper.cpp强制要求16kHzCHANNELS1FORMATpyaudio.paInt16# 一次性加载模型只执行一次 print(正在加载模型到内存...)modelModel(MODEL_PATH,n_threads8,use_gpuTrue,gpu_device0)print(✅ 模型加载完成常驻内存)print( 操作说明按住【鼠标左键】录音松开左键自动识别语音\n)# 录音全局状态is_recordingFalseaudio_frames[]streamNoneppyaudio.PyAudio()# 后台录音线程defmic_recorder():globalis_recording,audio_frames,stream streamp.open(formatFORMAT,channelsCHANNELS,rateSAMPLE_RATE,inputTrue,frames_per_buffer512)whileTrue:ifis_recording:try:datastream.read(512,exception_on_overflowFalse)audio_frames.append(data)exceptExceptionase:print(音频读取异常:,e)else:time.sleep(0.01)# 保存录音为wav临时文件defsave_audio_wav(output_pathtemp_rec.wav):wfwave.open(output_path,wb)wf.setnchannels(CHANNELS)wf.setsampwidth(p.get_sample_size(FORMAT))wf.setframerate(SAMPLE_RATE)wf.writeframes(b.join(audio_frames))wf.close()returnoutput_path# 语音识别函数模型已经在内存直接推理defspeech_recognize(wav_file):segmentsmodel.transcribe(wav_file,languageLANGUAGE)full_textforseginsegments:linef[{seg.start:.2f}s →{seg.end:.2f}s]{seg.text}print(line)full_textseg.textreturnfull_text# 启动录音后台线程record_threadthreading.Thread(targetmic_recorder,daemonTrue)record_thread.start()# 主循环监听鼠标左键try:whileTrue:left_downpyautogui.isPressed(left)ifleft_downandnotis_recording:# 按下左键开始录音清空缓存print(\n️ 开始录音 )is_recordingTrueaudio_frames.clear()elifnotleft_downandis_recording:# 松开左键停止录音执行识别print( 录音结束正在识别...)is_recordingFalsetemp_wavsave_audio_wav()resultspeech_recognize(temp_wav)print(f✅ 完整识别文本{result}\n)time.sleep(0.03)exceptKeyboardInterrupt:print(\n程序退出 CtrlC)finally:ifstream:stream.stop_stream()stream.close()p.terminate()总结vosk还是比阿里的逊色不少建议使用FunASR和whisper-cliwhisper-cli还可以自动识别标点符号