2023-04-26 / Task Form Decoration

qString: Type with Voice

translate qString: 音声で入力

Accepts voice input. Currently, it works only with Chrome and some other browsers. Press the Speech Recognition button to speak and characters will be entered. This is useful for reporting “facility inspection results” via smartphone. Note that the server-based recognition engine is used, so it does not work offline.

  • The Start button launches the “Speech Recognition service”.
  • Usage scenarios
    • Recording when both hands are occupied
    • Recording with a smart phone
    • Text input for people with hand disabilities
  • Behavior varies depending on OS and browser.
    • Timing of end of voice reception
    • Timing of confirmation decision
    • Efficiency of character conversion
  • The mode that return intermediate results currently does not work properly on smartphones.
    • user_recog.interimResults = true;
  • Unpronounced characters such as punctuation marks and line breaks can be entered using buttons.
    • It is also very effective to have a generative AI (such as “Chat GPT”) “proofread” the text in a downstream process.
    • Text typed by voice -> text proofread by AI
      • “press machine for production line
        the fuse on the control board has deteriorated we would like to consider replacing
        it there is a slight way inside the cylinder there is no problem but we would like to monitor its progress”
      • “Press machine for production line:
        The fuse on the control board has deteriorated, so we would like to consider replacing it.
        There is a slight wear inside the cylinder; although there is no problem at the moment, we would like to monitor its progress.”

Input / Output

  • ← (Microphone Device)
  • → qString (STRING_TEXTAREA) q_EquipmentProblem

Code Example

HTML/JavaScript (click to close)
<button type="button" onclick="user_stopRecog()">
 <span class="material-icons">stop_circle</span></button>
<button type="button" onclick="user_startRecog( 'en-US' )" class="user_startBtn">
(β) <span class="material-icons">mic</span> en-US</button>
<button type="button" onclick="user_startRecog( 'ja-JP' )" class="user_startBtn">
(β) <span class="material-icons">mic</span> ja-JP</button>

<div style="margin:5px 0px 3px 10px;">
<button type="button" style="min-height:30px; min-width:30px; border-radius: 15px;"
 onclick="user_insertChar( ', ' )"> , </button>
<button type="button" style="min-height:30px; min-width:30px; border-radius: 15px;"
 onclick="user_insertChar( '. ' )"> . </button>
<button type="button" style="min-height:30px; min-width:30px; border-radius: 15px;"
 onclick="user_insertChar( ': ' )"> : </button>
<button type="button" style="min-height:30px; min-width:30px; border-radius: 15px;"
 onclick="user_insertChar( '? ' )"> ? </button>
<button type="button" style="min-height:30px; min-width:30px; border-radius: 15px;"
 onclick="user_insertChar( '、' )"> 、</button>
<button type="button" style="min-height:30px; min-width:30px; border-radius: 15px;"
 onclick="user_insertChar( '。' )"> 。</button>
<button type="button" style="min-height:30px; min-width:30px; border-radius: 15px;"
 onclick="user_insertChar( '\n' )"> <span class="material-icons">keyboard_return</span></button>
</div>

<div id="user_speechStatus" style="padding:2px; margin:5px; border: 3px solid #FFC69B; "
>(SpeechAPI Status viewer...)</div>
<div id="user_firedCounter" style="margin:0px 10px; word-break: break-all;"> (no audio events fired) </div>

<script>
const user_qString = "q_EquipmentProblem"; // ★★★ EDIT Field Name ★★★
let user_strFiredCounter = '';
let user_strOriginal = '';

//// setup Web Speech API
SpeechRecognition = window.webkitSpeechRecognition || window.SpeechRecognition;
if( ! ('SpeechRecognition' in window) ) { console.error('speech-recognition not supported'); }
const user_recog = new SpeechRecognition();
      user_recog.interimResults = false; // interim results should be returned
      user_recog.continuous     = true; // continuous results are returned for each recognition
      // https://developer.mozilla.org/docs/Web/API/SpeechRecognition

//// Speaker action: start
function user_startRecog ( strLang ) {
  user_strFiredCounter += '>';
  user_strOriginal = qbpms.form.get( user_qString );
  user_recog.lang = strLang;
  user_recog.start();
  user_showMsg( "#user_speechStatus",            // ★ EDIT id ★
                " Recognition Started. mode: " + strLang );
  console.log(  " Recognition Started. mode: " + strLang );
  const btns = document.querySelectorAll( ".user_startBtn" );
        btns.forEach ( el => { el.disabled = true; el.style.cursor = "not-allowed"; } );
}

//// Speaker action: stop
function user_stopRecog () {
  user_strFiredCounter += '<';
  user_recog.stop(); // -> event handler property "onresult"
  user_showMsg( "#user_speechStatus",            // ★ EDIT id ★
                " Recognition Stopped." );
  console.log(  " Recognition Stopped." );
  const btns = document.querySelectorAll( ".user_startBtn" );
        btns.forEach ( el => { el.disabled = false; el.style.cursor = "pointer"; } );
}

//// Speaker action: insert character or linebreak
function user_insertChar ( strSuffix ) {
  user_strFiredCounter += '|';
  user_recog.stop(); // -> event handler property "onresult"
  console.log(  " inserted" );
  const btns = document.querySelectorAll( ".user_startBtn" );
        btns.forEach ( el => { el.disabled = false; el.style.cursor = "pointer"; } );
  let strTmp = qbpms.form.get( user_qString );
  // if ( ! strSuffix.endsWith('\n') ){ strTmp += ' '; };
  qbpms.form.set( user_qString, strTmp + strSuffix );
}

//// Show Interim Result to Msg, Append Final Result to qString
user_recog.onresult = (event) => {
  console.log( "--- Result Event fired (a word or phrase recognized) ! --- " );
  user_strFiredCounter += ( event.resultIndex % 10 ) + '';
  user_showMsg( "#user_firedCounter",            // ★ EDIT id ★
                 user_strFiredCounter + " " +
                 event.results.length
              );
  let strTmp = '';
  for ( let i = event.resultIndex; i < event.results.length; i++ ) {
    // English "event.resultIndex" on Android 12, even the index '0' changes frequently.
    if ( ! event.results[i].isFinal ) { // an interim result
      strTmp += event.results[i][0].transcript;
      user_showMsg( "#user_speechStatus", strTmp ); // ★ EDIT id ★
    } else { // when isFinal===true, joins all results
      let strFinal = '';
      for ( let j = 0; j < event.results.length; j++ ) {
        strFinal += event.results[j][0].transcript;
      }
      qbpms.form.set( user_qString, user_strOriginal + strFinal );
    }
  }
};

/**
 * Show a message in the specified HTML element by its ID
 * @param {string} id - The ID of the HTML element where the message will be shown.
 * @param {string} text - The message text to be displayed in the HTML element.
 */
function user_showMsg ( id, text ) { // eg "#user_speechStatus"
  let elMsgArea = document.querySelector ( id );
  while ( elMsgArea.firstChild ) { elMsgArea.removeChild ( elMsgArea.firstChild ); }
  elMsgArea.appendChild( document.createTextNode( text ) );
}
</script>
warning Freely modifiable HTML/JavaScript code, MIT License. No warranty of any kind.
(Decoration using JavaScript is only available in the Professional edition: M213)

Capture

Accepts voice input. Currently, it works only with Chrome and some other browsers. Press the Speech Recognition button to speak and characters will be entered. This is useful for reporting "facility inspection results" via smartphone. Note that the server-based recognition engine is used, so it does not work offline.

interimResults: true

open_in_new Online DEMO (public form)

Appendix

  • user_startRecog(strLang: string): void
    • Start the speech recognition with the specified language.
    • Parameters:
      • strLang: string – The language code for speech recognition.
    • Returns: void
  • user_stopRecog(): void
    • Stop the speech recognition.
    • Returns: void
  • user_insertChar(strSuffix: string): void
    • Insert a character or linebreak to the recognized text.
    • Parameters:
      • strSuffix: string – The character or linebreak to be inserted.
    • Returns: void
  • user_showMsg(id: string, text: string): void
    • Show a message in the specified HTML element by its ID.
    • Parameters:
      • id: string – The ID of the HTML element where the message will be shown.
      • text: string – The message text to be displayed in the HTML element.
  • user_recog.onresult(event: SpeechRecognitionEvent): void
    • An event handler for the speech recognition result event.
    • Parameters:
      • event: SpeechRecognitionEvent – The speech recognition event object.
    • Returns: void

See Also

OpenAI: Chat, Start
qNumber: Speech

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: