Multiline String, Remove Duplicate Lines
Extracts and deletes all duplicate rows. In forward order mode, the first row is retained; in reverse order mode, the last row is retained. Note that “ABC” and “Abc” are recognized as different columns. (case sensitive)
Configs for this Auto Step
- StrConfA1
- A1: Set Input Text *#{EL}
- BoolConfB1
- B1: Forward mode or Reverse mode
- SelectConfC1
- C1: Select STRING DATA for Output TSV Text (update)
- SelectConfC2
- C2: Select STRING DATA for Removed Lines (update)
Script (click to open)
// GraalJS Script (engine type: 2)
//////// START "main()" ////////////////////////////////////////////////////////////////
main();
function main(){
//// == Config Retrieving / 工程コンフィグの参照 ==
const strInput = configs.get ( "StrConfA1" ); /// REQUIRED
if( strInput === "" ){
throw new Error( "\n AutomatedTask ConfigError:" +
" Config {A1: String} is empty \n" );
}
const boolReverseMode = configs.getObject( "BoolConfB1" ); /// TOGGLE
// https://questetra.zendesk.com/hc/ja/articles/360024574471-R2300 "Boolean object"
if( boolReverseMode ){ // true
engine.log( " AutomatedTask Config: Reverse order Search");
}else{
engine.log( " AutomatedTask Config: Forward order Search");
}
const strPocketOutput = configs.getObject( "SelectConfC1" ); // NotRequired
const strPocketRemoved = configs.getObject( "SelectConfC2" ); // NotRequired
//// == Data Retrieving / ワークフローデータの参照 ==
// (Nothing. Retrieved via Expression Language in Config Retrieving)
//// == Calculating / 演算 ==
const arrInput = strInput.split( '\n' );
let arrOutput = [];
let arrRemoved = [];
let setUniqKeys = new Set();
// The Set object lets you store unique values of any type. Require ES2015(ES6)
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set
// https://developer.mozilla.org/ja/docs/Web/JavaScript/Reference/Global_Objects/Set
if ( boolReverseMode ) {
arrInput.reverse(); // [ "line0", "line1", "line2" ] ⇒ [ "line2", "line1", "line0" ]
}
for ( let i = 0; i < arrInput.length; i++ ) {
let strKey = arrInput[i];
// engine.log( " AutomatedTask StrKey: " + strKey );
if ( setUniqKeys.has ( strKey ) ) {
// engine.log( " AutomatedTask SetUniqKeys has: " + strKey + " (line: " + i + ")" );
arrRemoved.push ( strKey );
} else {
arrOutput.push ( strKey );
setUniqKeys.add ( strKey );
}
}
if ( boolReverseMode ) {
if ( arrOutput.length !== 0 ){
arrOutput.reverse(); // [ "line2", "line1", "line0" ] ⇒ [ "line0", "line1", "line2" ]
}
if ( arrRemoved.length !== 0 ){
arrRemoved.reverse();
}
}
let strOutput = arrOutput?.join( '\n' ) ?? "";
let strRemoved = arrRemoved?.join( '\n' ) ?? "";
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
//// == Data Updating / ワークフローデータへの代入 ==
if ( strPocketOutput !== null ) {
engine.setData ( strPocketOutput, strOutput );
}
if ( strPocketRemoved !== null ) {
engine.setData ( strPocketRemoved, strRemoved );
}
} //////// END "main()" /////////////////////////////////////////////////////////////////
/*
Notes:
- When the process reaches the automated step, the "Multiline text" will be automatically evaluated.
- In Forward mode, the first occurrence of a row is retained.
- And the second and subsequent occurrences are deleted.
- If the "Output Text" is empty, the string type data item is updated with empty characters.
- If the "delete line" is empty, the string type data item is updated with an empty string.
Appendix-en:
- In computer science, a `Set` is an abstract data type:
- that can store unique values, without any particular order.
- It is a computer implementation of the mathematical concept of a finite set.
- https://en.wikipedia.org/wiki/Set_(abstract_data_type)
- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set
Notes-ja:
- 案件が自動工程に到達した際、「複数行テキスト」が評価されます。
- 正順モード(Forward mode)の場合、最初に出現した行が保持され、2回目以降に出現する行は削除されます。
- もし「出力テキスト」が空の場合、文字列型データ項目は、空文字で更新されます。
- もし「削除行」が空の場合、文字列型データ項目は、空文字で更新されます。
Appendix-ja:
- `Set` とは、コンピュータプログラミングで用いられる抽象データ型の一種。
- 順序のないデータの集まり(有限集合)を表現する抽象データ型。
- 同一のデータは一つしか含まれないことが保証される。
- https://w.wiki/6DCr
- https://developer.mozilla.org/ja/docs/Web/JavaScript/Reference/Global_Objects/Set
*/
Download
- multiline-string-remove-duplicate-lines-2023.xml
- 2023-01-10 (C) Questetra, Inc. (MIT License)
(Installing Addon Auto-Steps are available only on the Professional edition.)
Notes
- When the process reaches the automated step, the “Multiline text” will be automatically evaluated.
- In Forward mode, the first occurrence of a row is retained.
- And the second and subsequent occurrences are deleted.
- If the “Output Text” is empty, the string type data item is updated with empty characters.
- If the “delete line” is empty, the string type data item is updated with an empty string.
Capture


Appendix
- In computer science, a
Set
is an abstract data type:- that can store unique values, without any particular order.
- It is a computer implementation of the mathematical concept of a finite set.
- https://en.wikipedia.org/wiki/Set_(abstract_data_type)
- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set