
Model Performance Parameters for Custom Models in Generative AI

Model Performance Parameters for Custom Models in Generative AI
When you create a custom model, OCI
Generative AI fine-tunes a pretrained model with the dataset.
 After fine-tuning is complete, you can access the model performance parameters on the model's
 details page to determine if your model has improved for your use case after fine-tuning on the
 dataset. The model performance parameters are computed on the validation dataset.
Model Performance Parameters for Generation Models
The following metrics apply to custom models.
Accuracy
Measures whether the generated tokens match the annotated tokens. For example, an
 accuracy of 0.9 means that 90 percent of the output tokens matched the tokens in the
 dataset.
Loss
Accuracy tells you how many predictions the model got wrong, and loss measures how wrong
 the generated outputs of a model are. A loss of 0 means that all outputs were perfect,
 while a large number indicates highly random outputs. Loss decreases as a model
 improves.
