{"id":150023,"date":"2023-08-23T17:07:04","date_gmt":"2023-08-23T08:07:04","guid":{"rendered":"https:\/\/support.questetra.com\/?p=150023"},"modified":"2023-08-25T11:27:38","modified_gmt":"2023-08-25T02:27:38","slug":"openai-ftjob-create-202308","status":"publish","type":"post","link":"https:\/\/support.questetra.com\/ja\/addons\/openai-ftjob-create-202308\/","title":{"rendered":"OpenAI #FtJob: \u751f\u6210"},"content":{"rendered":"\n

OpenAI #FtJob: \u751f\u6210<\/strong><\/h4>\n\n\n\n

translate<\/span> OpenAI #FtJob: Create<\/a><\/p>\n\n\n\n

MODEL \u3092\u30d5\u30a1\u30a4\u30f3\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u30b8\u30e7\u30d6\uff08FtJob\uff09\u3092\u958b\u59cb\u3055\u305b\u3001\u30b8\u30e7\u30d6\u306eID\u3092\u53d6\u5f97\u3057\u307e\u3059\u3002\u751f\u6210\u30b8\u30e7\u30d6\u306e\u5b8c\u4e86\u306b\u306f\u6570\u5206\u304b\u3089\u6570\u6642\u9593\u304b\u304b\u308a\u307e\u3059\u3002”\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u3055\u308c\u305f MODEL”\uff08FtMODL\uff09\u306f\u3001\u30b8\u30e7\u30d6\u5b8c\u4e86\u5f8c\u306b\u5229\u7528\u53ef\u80fd\u3068\u306a\u308a\u307e\u3059\u3002\u306a\u304a\u3001\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30d5\u30a1\u30a4\u30eb\uff08\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\uff09\u306f\u4e8b\u524d\u306b\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3057\u3066\u304a\u304f\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n\n\n\n

\"Auto<\/figure>\n\n\n\n
\n
Configs for this Auto Step<\/summary>
\n
\n
AuthzConfU1<\/dt>
U1: HTTP\u8a8d\u8a3c\u8a2d\u5b9a\u3092\u9078\u629e\u3057\u3066\u304f\u3060\u3055\u3044\uff08Secret API Key \uff20\u30c8\u30fc\u30af\u30f3\u76f4\u63a5\u6307\u5b9a\uff09 *<\/span><\/dd>\n
StrConfA1<\/dt>
A1: \u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\uff08\u5b66\u7fd2\uff09\u30d5\u30a1\u30a4\u30eb\u306eID\u3092\u30bb\u30c3\u30c8\u3057\u3066\u304f\u3060\u3055\u3044 *<\/span>#{EL}<\/sup><\/span><\/dd>\n
SelectConfB1<\/dt>
B1: Job ID \u304c\u683c\u7d0d\u3055\u308c\u308b\u6587\u5b57\u5217\u578b\u30c7\u30fc\u30bf\u9805\u76ee\u3092\u9078\u629e\u3057\u3066\u304f\u3060\u3055\u3044\uff08\u66f4\u65b0\uff09 *<\/span><\/dd>\n<\/div>\n
StrConfU2<\/dt>
U2: OpenAI Organization ID \u3092\u30bb\u30c3\u30c8\u3057\u3066\u304f\u3060\u3055\u3044\uff08”org-xxxx”\uff09#{EL}<\/sup><\/span><\/dd>\n
StrConfA2<\/dt>
A2: \u691c\u8a3c\u30d5\u30a1\u30a4\u30eb\u306eID\u3092\u30bb\u30c3\u30c8\u3057\u3066\u304f\u3060\u3055\u3044#{EL}<\/sup><\/span><\/dd>\n
StrConfA3<\/dt>
A3: \u30d9\u30fc\u30b9\u3068\u306a\u308b MODEL NAME\/ID \u3092\u30bb\u30c3\u30c8\u3057\u3066\u304f\u3060\u3055\u3044 *<\/span>#{EL}<\/sup><\/span><\/dd>\n
StrConfA4<\/dt>
A4: Fine-Tuned MODEL \u540d\u306b\u8ffd\u52a0\u3057\u305f\u3044 SUFFIX \u3092\u30bb\u30c3\u30c8\u3057\u3066\u304f\u3060\u3055\u3044#{EL}<\/sup><\/span><\/dd>\n<\/div>\n\n<\/dl>\n<\/div><\/details><\/div>\n<\/div>\n\n\n\n
\n
Script (click to open)<\/summary>
\n
\/\/ GraalJS standard mode Script Script (engine type: 3)\n\/\/ cf. 'engine type: 2': "GraalJS Nashorn compatible mode" (renamed from "GraalJS" at 20230526)\n\n\n\/\/\/\/\/\/\/\/ START "main()" \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\n\nmain();\nfunction main(){ \n\n\/\/\/\/\/\/ == Config Retrieving \/ \u5de5\u7a0b\u30b3\u30f3\u30d5\u30a3\u30b0\u306e\u53c2\u7167 ==\nconst strAuthzSetting     = configs.get( "AuthzConfU1" );             \/\/\/ REQUIRED\n  engine.log( " AutomatedTask Config: Authz Setting: " + strAuthzSetting );\nconst strOrgId            = configs.get( "StrConfU2" );               \/\/ NotRequired\n  engine.log( " AutomatedTask Config: OpenAI-Organization: " + strOrgId );\n\nconst strFileIdTraining   = configs.get( "StrConfA1" );               \/\/\/ REQUIRED\nconst strFileIdValidation = configs.get( "StrConfA2" );               \/\/ NotRequired\nconst strBaseModel        = configs.get( "StrConfA3" );               \/\/\/ REQUIRED (20230822)\nconst strSuffix           = configs.get( "StrConfA4" );               \/\/ NotRequired\n\nconst strPocketJobId      = configs.getObject( "SelectConfB1" );      \/\/\/ REQUIRED\n\n\n\n\/\/\/\/\/\/ == Data Retrieving \/ \u30ef\u30fc\u30af\u30d5\u30ed\u30fc\u30c7\u30fc\u30bf\u306e\u53c2\u7167 ==\n\/\/ (Nothing. Retrieved via Expression Language in Config Retrieving)\n\n\n\n\/\/\/\/\/\/ == Calculating \/ \u6f14\u7b97 ==\n\n\/\/\/\/ OpenAI API > Documentation > API REFERENCE > Fine-tunes > Create fine-tuning job\n\/\/\/\/ https:\/\/platform.openai.com\/docs\/api-reference\/fine-tuning\/create\n\n\/\/\/ prepare json\nlet strJson = {};\n    strJson.training_file = strFileIdTraining;\n    if ( strFileIdValidation !== "" ){\n      strJson.validation_file = strFileIdValidation;\n    }\n    strJson.model = strBaseModel;   \/\/ eg. "gpt-3.5-turbo" (asof 20230822)\n    if ( strSuffix !== "" ){\n      strJson.suffix = strSuffix;\n    }\n\n\/\/\/ prepare request1\nlet request1Uri = "https:\/\/api.openai.com\/v1\/fine_tuning\/jobs"; \/\/ updated 20230822\nlet request1 = httpClient.begin(); \/\/ HttpRequestWrapper\n    request1 = request1.authSetting( strAuthzSetting ); \/\/ with "Authorization: Bearer XX"\n    if ( strOrgId !== "" ){\n      request1 = request1.header( "OpenAI-Organization", strOrgId );\n    }\n    request1 = request1.body( JSON.stringify( strJson ), "application\/json" );\n\n\/\/\/ try request1\nconst response1     = request1.post( request1Uri ); \/\/ HttpResponseWrapper\nengine.log( " AutomatedTask ApiRequest1 Start: " + request1Uri );\nconst response1Code = response1.getStatusCode() + ""; \/\/ JavaNum to string\nconst response1Body = response1.getResponseAsString();\nengine.log( " AutomatedTask ApiResponse1 Status: " + response1Code );\nif( response1Code !== "200"){\n  throw new Error( "\\n AutomatedTask UnexpectedResponseError: " +\n                    response1Code + "\\n" + response1Body + "\\n" );\n}\n\n\/* engine.log( response1Body ); \/\/ debug\n{\n  "object":"fine_tuning.job",\n  "id":"ftjob-EcRIIXZMnZzbEqyrFvJ2vYLZ",\n  "model":"gpt-3.5-turbo-0613",\n  "created_at":1692772480,\n  "finished_at":null,\n  "fine_tuned_model":null,\n  "organization_id":"org-xxxxxyyyyyzzzzzXXXXXYYYY",\n  "result_files":[],\n  "status":"created",\n  "validation_file":null,\n  "training_file":"file-UaXSB3M90mpinC3pH3ZGiTiJ",\n  "hyperparameters":{"n_epochs":5},\n  "trained_tokens":null\n}\n*\/\n\n\/\/\/ parse response1\nconst response1Obj = JSON.parse( response1Body );\n\n\n\n\/\/\/\/\/\/ == Data Updating \/ \u30ef\u30fc\u30af\u30d5\u30ed\u30fc\u30c7\u30fc\u30bf\u3078\u306e\u4ee3\u5165 ==\nif( strPocketJobId !== null ){\n  engine.setData( strPocketJobId, response1Obj.id );\n}\n\n\n} \/\/\/\/\/\/\/\/ END "main()" \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\n\n\n\/*\nNotes:\n- This "Automated Step" will start to generate a Fine-Tuned MODEL on the OpenAI server.\n    - It is possible to specify the base MODEL.\n    - The confirmation step is placed downstream. (OpenAI #FtJob: Retrieve)\n- If you place this "Automated Step" in the Workflow diagram, the request will be automatically sent every time the process token arrives.\n    - A request is automatically sent to the OpenAI API server. (REST API)\n    - The response from the OpenAI API server is automatically parsed.\n\nAPPENDIX\n- To activate a Workflow App that includes this Automated Step, "HTTP Authz Setting" is required\n    - Obtain a "Secret API Key" in advance.\n    - Set the key as the communication token in "Token Fixed Value"\n- This automated step does not support the following parameters.\n     - `n_epochs`\n     - `batch_size`\n     - `learning_rate_multiplier`\n     - `prompt_loss_weight`\n     - `compute_classification_metrics`\n     - `classification_n_classes`\n     - `classification_positive_class`\n     - `classification_betas`\n- Check the OpenAI document for the MODEL that can be specified as the Base MODEL for fine tuning.\n    - https:\/\/platform.openai.com\/docs\/guides\/fine-tuning\/what-models-can-be-fine-tuned\n    - Asof 202307: Fine-tuning is currently only available for the following base models: `davinci`, `curie`, `babbage`, and `ada`. We are working on safely enabling fine-tuning for GPT-4 and GPT-3.5 Turbo and expect this feature to be available later this year.\n    - Asof 202308: Fine-tuning is currently available for the following models:\n        - gpt-3.5-turbo-0613 (recommended)\n        - babbage-002\n        - davinci-002\n- When setting SUFFIX, set it within 40 characters.\n     - eg: "custom-model-name"\n     - MODEL name: "ada:ft-your-org:custom-model-name-2022-02-15-04-21-04"\n- The format of the training data has also changed. (2023-08-22)\n    - https:\/\/platform.openai.com\/docs\/guides\/fine-tuning\/preparing-your-dataset\n    - 20230822~: `{"messages": [{"role": "system", "content": "\u25cf"}, {"role": "user", "content": "\u25cf"}, {"role": "assistant", "content": "\u25cf"}]}`\n    - ~20230822: `{"prompt": "<prompt text>", "completion": "<ideal generated text>"}`\n    - error to: `"message": "File file-xxxxxyyyyyzzzzzXXXXXYYYY format is not compatible with model gpt-3.5-turbo-0613"`\n    - At least 10 examples are required. For clear improvements on 50 to 100 training examples typically.\n    - error to: `"message": "file-xxxxxyyyyyzzzzzXXXXXYYYY has 4 example(s), but must have at least 10 examples"`\n\n\nNotes-ja:\n- \u3053\u306e\uff3b\u81ea\u52d5\u5de5\u7a0b\uff3d\u306f\u3001OpenAI \u30b5\u30fc\u30d0\u4e0a\u306e Fine-Tuned MODEL \u751f\u6210\u3092\u958b\u59cb\u3055\u305b\u307e\u3059\u3002\n    - \u30d9\u30fc\u30b9\u3068\u306a\u308b MODEL \u3092\u6307\u5b9a\u3067\u304d\u307e\u3059\u3002\n    - \u30d5\u30a1\u30a4\u30f3\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\uff08\u5fae\u8abf\u6574\uff09\u306e\u6210\u679c\u306f\u3001\u4e0b\u6d41\u306b\u9032\u6357\u78ba\u8a8d\u5de5\u7a0b\u3092\u914d\u7f6e\u3057\u78ba\u8a8d\u3057\u307e\u3059\u3002\uff08OpenAI #FtJob: \u9032\u6357\u78ba\u8a8d\uff09\n- \u3053\u306e\uff3b\u81ea\u52d5\u5de5\u7a0b\uff3d\u3092\u30ef\u30fc\u30af\u30d5\u30ed\u30fc\u56f3\u306b\u914d\u7f6e\u3059\u308c\u3070\u3001\u6848\u4ef6\u304c\u5230\u9054\u3059\u308b\u5ea6\u306b\u30ea\u30af\u30a8\u30b9\u30c8\u304c\u81ea\u52d5\u9001\u4fe1\u3055\u308c\u307e\u3059\u3002\n    - OpenAI API \u30b5\u30fc\u30d0\u306b\u5bfe\u3057\u3066\u30ea\u30af\u30a8\u30b9\u30c8\u304c\u81ea\u52d5\u9001\u51fa\u3055\u308c\u307e\u3059\u3002\uff08REST API\u901a\u4fe1\uff09\n    - OpenAI API \u30b5\u30fc\u30d0\u304b\u3089\u306e\u30ec\u30b9\u30dd\u30f3\u30b9\u304c\u81ea\u52d5\u4fdd\u5b58\u89e3\u6790\u3055\u308c\u307e\u3059\u3002\n\nAPPENDIX-ja\n- \u3053\u306e\uff3b\u30a2\u30c9\u30aa\u30f3\u81ea\u52d5\u5de5\u7a0b\uff3d\u3092\u542b\u3080\u30ef\u30fc\u30af\u30d5\u30ed\u30fc\u30a2\u30d7\u30ea\u3092\u904b\u7528\u3059\u308b\u306b\u306f\uff3bHTTP \u8a8d\u8a3c\u8a2d\u5b9a\uff3d\u304c\u5fc5\u8981\u3067\u3059\u3002\n    - \u3042\u3089\u304b\u3058\u3081 "Secret API Key" \u3092\u53d6\u5f97\u3057\u3066\u304a\u3044\u3066\u304f\u3060\u3055\u3044\u3002\n    - "Secret API Key" \u3092\u901a\u4fe1\u30c8\u30fc\u30af\u30f3\u3068\u3057\u3066\u30bb\u30c3\u30c8\u3057\u307e\u3059\u3002\uff3b\u30c8\u30fc\u30af\u30f3\u76f4\u63a5\u6307\u5b9a\uff3d\n- \u3053\u306e\u81ea\u52d5\u5de5\u7a0b\u306f\u3001\u4ee5\u4e0b\u306e\u30d1\u30e9\u30e1\u30fc\u30bf\u306b\u5bfe\u5fdc\u3057\u3066\u3044\u307e\u305b\u3093\u3002\n    - `n_epochs`\n    - `batch_size`\n    - `learning_rate_multiplier`\n    - `prompt_loss_weight`\n    - `compute_classification_metrics`\n    - `classification_n_classes`\n    - `classification_positive_class`\n    - `classification_betas`\n- \u30d5\u30a1\u30a4\u30f3\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u306e Base MODEL \u3068\u3057\u3066\u6307\u5b9a\u53ef\u80fd\u306a MODEL \u306b\u3064\u3044\u3066\u306f OpenAI \u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u3092\u53c2\u7167\u304f\u3060\u3055\u3044\u3002\n    - https:\/\/platform.openai.com\/docs\/guides\/fine-tuning\/what-models-can-be-fine-tuned\n    - Asof 202307: Fine-tuning is currently only available for the following base models: `davinci`, `curie`, `babbage`, and `ada`. We are working on safely enabling fine-tuning for GPT-4 and GPT-3.5 Turbo and expect this feature to be available later this year.\n    - Asof 202308: Fine-tuning is currently available for the following models:\n        - gpt-3.5-turbo-0613 (recommended)\n        - babbage-002\n        - davinci-002\n- SUFFIX \u3092\u8a2d\u5b9a\u3059\u308b\u5834\u5408\u306f40\u6587\u5b57\u4ee5\u5185\u3067\u8a2d\u5b9a\u3057\u307e\u3059\u3002\n    - eg: "custom-model-name"\n    - MODEL name: "ada:ft-your-org:custom-model-name-2022-02-15-04-21-04"\n- \u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30c7\u30fc\u30bf\u306e\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u3082\u5909\u66f4\u3055\u308c\u3066\u3044\u307e\u3059\u3002 (2023-08-22)\n    - https:\/\/platform.openai.com\/docs\/guides\/fine-tuning\/preparing-your-dataset\n    - 20230822~: `{"messages": [{"role": "system", "content": "\u25cf"}, {"role": "user", "content": "\u25cf"}, {"role": "assistant", "content": "\u25cf"}]}`\n    - ~20230822: `{"prompt": "<prompt text>", "completion": "<ideal generated text>"}`\n    - error to: `"message": "File file-xxxxxyyyyyzzzzzXXXXXYYYY format is not compatible with model gpt-3.5-turbo-0613"`\n    - \u5c11\u306a\u304f\u3068\u308210\u500b\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30c7\u30fc\u30bf\u304c\u5fc5\u8981\u3067\u3059\u3002\u901a\u5e38\u300150\uff5e100\u500b\u306e\u30c7\u30fc\u30bf\u3067\u660e\u78ba\u306a\u6539\u5584\u304c\u671f\u5f85\u3055\u308c\u307e\u3059\u3002\n        - error to: `"message": "file-xxxxxyyyyyzzzzzXXXXXYYYY has 4 example(s), but must have at least 10 examples"`\n*\/\n\n<\/code><\/pre><\/div>\n<\/div><\/details><\/div>\n<\/div>\n\n\n\n
Download<\/span><\/h3>\n\n\n\n