{"id":141140,"date":"2023-01-16T17:03:54","date_gmt":"2023-01-16T08:03:54","guid":{"rendered":"https:\/\/support.questetra.com\/?p=141140"},"modified":"2023-01-16T17:38:44","modified_gmt":"2023-01-16T08:38:44","slug":"text-files-convert-character-encoding-2023","status":"publish","type":"post","link":"https:\/\/support.questetra.com\/en\/addons\/text-files-convert-character-encoding-2023\/","title":{"rendered":"Text Files, Convert Character Encoding"},"content":{"rendered":"\n

Text Files, Convert Character Encoding<\/strong><\/h4>\n\n\n\n

Text \u30d5\u30a1\u30a4\u30eb, \u6587\u5b57\u30a8\u30f3\u30b3\u30fc\u30c7\u30a3\u30f3\u30b0\u306e\u5909\u63db<\/a><\/p>\n\n\n\n

Converts text files charset (Coded Character Set). For example, converts UTF-8 encoding to Shift_JIS or UTF-16. If multiple files are attached, all will be converted according to the same rules.<\/p>\n\n\n\n

\"Auto<\/figure>\n\n\n\n
\n
Configs for this Auto Step<\/summary>
\n
\n
SelectConfA1<\/dt>
A1: Select FILE DATA for Original Text Files *<\/span><\/dd>\n
StrConfA2<\/dt>
A2: Set Original Charset (eg “UTF-8” )#{EL}<\/sup><\/span><\/dd>\n
StrConfB1<\/dt>
B1: Set New Charset (eg “UTF-16” )#{EL}<\/sup><\/span><\/dd>\n
SelectConfB2<\/dt>
B2: Select FILE DATA that stores New Text Files (append) *<\/span><\/dd>\n\n<\/dl>\n<\/div><\/details><\/div>\n<\/div>\n\n\n\n
\n
Script (click to open)<\/summary>
\n
\/\/ GraalJS Script (engine type: 2)\n\n\/\/\/\/\/\/\/\/ START "main()" \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\nmain();\nfunction main(){ \n\n\/\/\/\/ == Config Retrieving \/ \u5de5\u7a0b\u30b3\u30f3\u30d5\u30a3\u30b0\u306e\u53c2\u7167 ==\nconst filesPocketInput    = configs.getObject( "SelectConfA1" );  \/\/\/ REQUIRED \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\n  let filesInput          = engine.findData( filesPocketInput );  \/\/ java.util.ArrayList\n  if( filesInput        === null ) {\n    throw new Error( "\\n AutomatedTask UnexpectedFileError:" +\n                     " No File {A1} is attached \\n" );\n  }else{\n    engine.log( " AutomatedTask FilesArray {A1}: " +\n                filesInput.size() + " files" );\n  }\nlet   strInputCharset     = configs.get( "StrConfA2" );           \/\/ NotRequired \/\/\/\/\/\/\/\/\/\/\/\/\/\n  if( strInputCharset   === "" ){\n      strInputCharset     = "UTF-8";\n  }\nlet   strOutputCharset    = configs.get( "StrConfB1" );           \/\/ NotRequired \/\/\/\/\/\/\/\/\/\/\/\/\/\n  if( strOutputCharset  === "" ){\n      strOutputCharset    = "UTF-8";\n  }\nconst filesPocketOutput   = configs.getObject( "SelectConfB2" );  \/\/\/ REQUIRED \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\n  let filesOutput         = engine.findData( filesPocketOutput ); \/\/ java.util.ArrayList\n  if( filesOutput       === null ) {\n    engine.log( " AutomatedTask FilesArray {B2}: (empty)" );\n    filesOutput           = new java.util.ArrayList();\n  }else{\n    engine.log( " AutomatedTask FilesArray {B2}: " +\n                filesOutput.size() + " files" );\n  }\n\n\n\/\/\/\/ == Data Retrieving \/ \u30ef\u30fc\u30af\u30d5\u30ed\u30fc\u30c7\u30fc\u30bf\u306e\u53c2\u7167 ==\n\/\/ (Nothing. Retrieved via Expression Language in Config Retrieving)\n\n\n\/\/\/\/ == Calculating \/ \u6f14\u7b97 ==\nconst numFilesInput = filesInput.size() - 0;\nconst regMime = \/\\w+\\\/[-\\w.+]+\/;                                                \/\/ +++ in v2023\u2605\n\nfor( let i = 0; i < numFilesInput; i++ ){\n  const strInputFileName = filesInput.get(i).getName() + "";\n  const strInputFileSize = filesInput.get(i).getLength() + " bytes";\n\n  \/\/  const strInputFileMime = filesInput.get(i).getContentType();              \/\/ --- in v2023\u2605\n  const arrInputFileMime = filesInput.get(i).getContentType().match( regMime ); \/\/ +++ in v2023\u2605\n  if( arrInputFileMime === null ){                                              \/\/ +++ in v2023\u2605\n    throw new Error( "\\n AutomatedTask RuntimeError:" +\n                     " MIME Type of the input, unreferenable \\n" );\n  }\n  const strInputFileMime = arrInputFileMime[0];                                 \/\/ +++ in v2023\u2605\n\n  let strInputText = "";\n  let numLineCounter = 0;\n  fileRepository.readFile( filesInput.get(i), strInputCharset, function(line) {\n  \/\/ com.questetra.bpms.core.event.scripttask.FileRepositoryWrapper\n  \/\/ https:\/\/questetra.zendesk.com\/hc\/en-us\/articles\/360024574471-R2300#FileRepositoryWrapper\n      strInputText += line + '\\n';\n      numLineCounter ++;\n  });\n  engine.log( " AutomatedTask FileLoaded: " + strInputFileName + " (" + strInputFileMime + ")" );\n  engine.log( " AutomatedTask: " + strInputFileSize + " \/ " + numLineCounter + " lines" );\n  engine.log( " AutomatedTask FileOutput as: " +\n                strInputFileMime + "; charset=" + strOutputCharset );           \/\/ +++ in v2023\u2605\n\n  filesOutput.add(\n    new com.questetra.bpms.core.event.scripttask.NewQfile(\n      strInputFileName,\n      strInputFileMime + "; charset=" + strOutputCharset,\n      strInputText\n    )\n  );\n}\n\n\n\/\/\/\/ == Data Updating \/ \u30ef\u30fc\u30af\u30d5\u30ed\u30fc\u30c7\u30fc\u30bf\u3078\u306e\u4ee3\u5165 ==\nengine.setData( filesPocketOutput, filesOutput );\n\n} \/\/\/\/\/\/\/\/ END "main()" \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\n\n\n\/*\nNotes:\n- Used when incorporating "Step in which Text file Encoding is automatically changed" in the workflow.\n    - Charset of Text file is automatically changed when the process reaches this automated task.\n- The file name of the output file will be the same as the input file.\n    - The line feed code is `LF`.\n- Converts according to the specified Encodings.\n    - No auto-detect feature.\n    - If not specified, the default Encoding is `UTF-8`.\n\nAPPENDIX:\n- `UTF-8`\n    - Compactly encodes more than 1 million Unicode characters in the world with 1 to 4 bytes.\n    - It became the most common character code in 2008 and is used in 97% of web pages as of 2021.\n    - UTF-8 is superset of US-ASCII (single-byte characters). (upward compatible)\n        - That is, ASCII files are also UTF-8 files. (US-ASCII is a subset of UTF-8)\n        - Similarly, ASCII files are also Shift_JIS files.\n- `UTF-16`\n    - Encodes over 1 million Unicode characters in the world with 2-4 bytes.\n    - If there are many Asian characters such as Japanese and Chinese, encode them compactly.\n- Another encoding\n    - `charset=UTF-16` (Unicode [characters around the world])\n    - `charset=UTF-16BE` (Unicode [characters around the world])\n    - `charset=UTF-16LE` (Unicode [characters around the world])\n    - `charset=UTF-32` (Unicode [characters around the world])\n    - `charset=x-UTF-32LE-BOM` (Unicode [characters around the world])\n    - `charset=ISO-8859-1` (Western language characters)\n    - `charset=Shift_JIS` (Japanese characters)\n    - `charset=Big5` (Traditional Chinese characters)\n    - `charset=GB2312` (Simplified Chinese EUC characters)\n    - `charset=GBK` (Simplified Chinese GB characters)\n    - `charset=KOI8-R` (Russian)\n    - In addition, "UTF8B (UTF-8 with BOM)" cannot be output. (File for Windows \/ pray for its eradication)\n    - https:\/\/docs.oracle.com\/javase\/9\/intl\/supported-encodings.htm\n- Example Values of `.getContentType()` \/\/ +++ in v2023\u2605\n    - "text\/plain"\n    - "text\/html;charset=UTF-8"\n    - "application\/json; q=0.2 charset=utf8"\n    - "text\/html; charset=ISO-8859-4"\n- Content-Type\n    - https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/HTTP\/Headers\/Content-Type\n    - https:\/\/developer.mozilla.org\/ja\/docs\/Web\/HTTP\/Headers\/Content-Type\n    - https:\/\/httpwg.org\/specs\/rfc9110.html#field.content-type\n- Media Types\n    - https:\/\/www.iana.org\/assignments\/media-types\/media-types.xhtml\n\n\nNotes-ja:\n- \u30ef\u30fc\u30af\u30d5\u30ed\u30fc\u5185\u306b\u300cText\u30d5\u30a1\u30a4\u30ebEncoding\u304c\u81ea\u52d5\u7684\u306b\u5909\u66f4\u3055\u308c\u308b\u5de5\u7a0b\u300d\u3092\u7d44\u307f\u8fbc\u3080\u969b\u306b\u5229\u7528\u3057\u307e\u3059\u3002\n    - \u6848\u4ef6\u304c\u81ea\u52d5\u51e6\u7406\u5de5\u7a0b\u306b\u5230\u9054\u3057\u305f\u969b\u3001Text\u30d5\u30a1\u30a4\u30eb\u306eCharset\u304c\u81ea\u52d5\u7684\u306b\u5909\u66f4\u3055\u308c\u307e\u3059\u3002\n- \u51fa\u529b\u30d5\u30a1\u30a4\u30eb\u306e\u30d5\u30a1\u30a4\u30eb\u540d\u306f\u3001\u5165\u529b\u30d5\u30a1\u30a4\u30eb\u3068\u540c\u3058\u30d5\u30a1\u30a4\u30eb\u540d\u306b\u306a\u308a\u307e\u3059\u3002\n    - \u6539\u884c\u30b3\u30fc\u30c9\u306f `LF` \u3067\u3059\u3002\n- \u6307\u5b9a\u3055\u308c\u305f Encoding \u306b\u5f93\u3063\u3066\u5909\u63db\u3057\u307e\u3059\u3002\n    - \u81ea\u52d5\u5224\u5225\u6a5f\u80fd\u306f\u3042\u308a\u307e\u305b\u3093\u3002\n    - \u672a\u6307\u5b9a\u306e\u5834\u5408\u3001\u30c7\u30d5\u30a9\u30eb\u30c8\u306e Encoding \u306f `UTF-8` \u3067\u3059\u3002\n\nAPPENDIX-ja:\n- `UTF-8`\n    - \u4e16\u754c100\u4e07\u7a2e\u4ee5\u4e0a\u306eUnicode\u6587\u5b57\u3092\u30011\uff5e4\u30d0\u30a4\u30c8\u3067\u30b3\u30f3\u30d1\u30af\u30c8\u306b\u30a8\u30f3\u30b3\u30fc\u30c9\u3057\u307e\u3059\u3002\n    - 2008\u5e74\u306b\u6700\u3082\u4e00\u822c\u7684\u306a\u6587\u5b57\u30b3\u30fc\u30c9\u3068\u306a\u308a\u30012021\u5e74\u6642\u70b9\u306797%\u306e\u30a6\u30a7\u30d6\u30da\u30fc\u30b8\u3067\u5229\u7528\u3055\u308c\u3066\u3044\u307e\u3059\u3002\n    - UTF-8 \u306f US-ASCII\uff081\u30d0\u30a4\u30c8\u6587\u5b57\uff09\u306e\u4e0a\u4f4d\u4e92\u63db\u3067\u3059\u3002\uff08US-ASCII \u306f UTF-8 \u306e\u30b5\u30d6\u30bb\u30c3\u30c8\u3067\u3059\uff09\n        - \u3059\u306a\u308f\u3061 ASCII \u30d5\u30a1\u30a4\u30eb\u306f UTF-8 \u30d5\u30a1\u30a4\u30eb\u3067\u3082\u3042\u308a\u307e\u3059\u3002\n        - \u540c\u69d8\u306b ASCII \u30d5\u30a1\u30a4\u30eb\u306f Shift_JIS \u30d5\u30a1\u30a4\u30eb\u3067\u3082\u3042\u308a\u307e\u3059\u3002\n- `UTF-16`\n    - \u4e16\u754c100\u4e07\u7a2e\u4ee5\u4e0a\u306eUnicode\u6587\u5b57\u3092\u30012\uff5e4\u30d0\u30a4\u30c8\u3067\u30a8\u30f3\u30b3\u30fc\u30c9\u3057\u307e\u3059\u3002\n    - \u65e5\u672c\u8a9e\u3084\u4e2d\u56fd\u8a9e\u306a\u3069\u306e\u30a2\u30b8\u30a2\u6587\u5b57\u304c\u591a\u3044\u5834\u5408\u306f\u3001\u30b3\u30f3\u30d1\u30af\u30c8\u306b\u30a8\u30f3\u30b3\u30fc\u30c9\u3057\u307e\u3059\u3002\n- \u305d\u306e\u4ed6\u306e\u30a8\u30f3\u30b3\u30fc\u30c7\u30a3\u30f3\u30b0\n    - `charset=UTF-16` (Unicode\uff3b\u4e16\u754c\u4e2d\u306e\u6587\u5b57\uff3d)\n    - `charset=UTF-16BE` (Unicode\uff3b\u4e16\u754c\u4e2d\u306e\u6587\u5b57\uff3d)\n    - `charset=UTF-16LE` (Unicode\uff3b\u4e16\u754c\u4e2d\u306e\u6587\u5b57\uff3d)\n    - `charset=UTF-32` (Unicode\uff3b\u4e16\u754c\u4e2d\u306e\u6587\u5b57\uff3d)\n    - `charset=x-UTF-32LE-BOM` (Unicode\uff3b\u4e16\u754c\u4e2d\u306e\u6587\u5b57\uff3d)\n    - `charset=ISO-8859-1` (\u30e8\u30fc\u30ed\u30c3\u30d1\u8a00\u8a9e\u306e\u6587\u5b57)\n    - `charset=Shift_JIS` (\u65e5\u672c\u8a9e\u306e\u6587\u5b57)\n    - `charset=Big5` (\u7e41\u4f53\u4e2d\u56fd\u8a9e\u306e\u6587\u5b57)\n    - `charset=GB2312` (\u7c21\u4f53\u4e2d\u56fd\u8a9eEUC\u6587\u5b57)\n    - `charset=GBK` (\u7c21\u4f53\u4e2d\u56fd\u8a9eGB\u306e\u6587\u5b57)\n    - `charset=KOI8-R` (\u30ed\u30b7\u30a2\u8a9e)\n    - \u306a\u304a "UTF8B (BOM\u4ed8 UTF-8)" \u306f\u51fa\u529b\u3067\u304d\u307e\u305b\u3093\u3002\uff08Windows\u7528\u30d5\u30a1\u30a4\u30eb\uff0f\u305d\u306e\u64b2\u6ec5\u3092\u7948\u5ff5\uff09\n    - https:\/\/docs.oracle.com\/javase\/9\/intl\/supported-encodings.htm\n    - https:\/\/docs.oracle.com\/javase\/jp\/9\/intl\/supported-encodings.htm\n- Example Values of `.getContentType()` \/\/ +++ in v2023\u2605\n    - "text\/plain"\n    - "text\/html;charset=UTF-8"\n    - "application\/json; q=0.2 charset=utf8"\n    - "text\/html; charset=ISO-8859-4"\n- Content-Type\n    - https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/HTTP\/Headers\/Content-Type\n    - https:\/\/developer.mozilla.org\/ja\/docs\/Web\/HTTP\/Headers\/Content-Type\n    - https:\/\/httpwg.org\/specs\/rfc9110.html#field.content-type\n- Media Types\n    - https:\/\/www.iana.org\/assignments\/media-types\/media-types.xhtml\n*\/\n\n<\/code><\/pre><\/div>\n<\/div><\/details><\/div>\n<\/div>\n\n\n\n
Download<\/span><\/h3>\n\n\n\n