eval: add Oracle ERP Cloud workflow and terminology eval#1654
Open
karthikchundi-commits wants to merge 1 commit intoopenai:mainfrom
Open
eval: add Oracle ERP Cloud workflow and terminology eval#1654karthikchundi-commits wants to merge 1 commit intoopenai:mainfrom
karthikchundi-commits wants to merge 1 commit intoopenai:mainfrom
Conversation
Adds oracle_erp_workflow — 30 samples testing LLM knowledge of Oracle ERP Cloud across Procurement, HCM, General Ledger, Fixed Assets, SCM, EPM Planning, and Oracle Integration Cloud. Samples cover: - Core process flows (procure-to-pay, three-way match, order-to-cash) - Document types (BPA, transfer order, RMA, work order, mass addition) - Approval workflows (PO submission, budget promotion, invoice holds) - Module-specific terminology (substitution variables, approval units, work relationships, element entries, account combinations) Questions are drawn from real enterprise deployment scenarios where LLMs are increasingly being used for ERP process guidance, workflow automation, and natural-language querying of ERP systems.
karthikchundi-commits
requested review from
andrew-openai,
etr2460 and
katyhshi
as code owners
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eval description
oracle_erp_workflow— 30 samples testing LLM knowledge of Oracle ERP Cloud workflows, document types, approval hierarchies, and module-specific terminology.Motivation
LLMs are increasingly being used in enterprise ERP contexts — for process guidance, workflow automation, natural-language querying, and agentic ERP integrations. Oracle ERP Cloud (Fusion) is one of the most widely deployed enterprise platforms globally. There is currently no eval in the registry covering this domain.
Modules covered
Eval structure
FuzzyMatch— answers are short phrases or single terms with clear ground truthevals/registry/data/oracle_erp_workflow/samples.jsonlevals/registry/evals/oracle_erp_workflow.yamlContributor background
16 years of hands-on Oracle ERP and integration experience across Procurement, HCM, Financials, and OIC; currently pursuing a Master's in AI/ML.