# Genie Accuracy

Genie Accuracy APIs provide a method to evaluate and measure the accuracy of LLM inference results. The accuracy feature supports computing perplexity metrics on a given dataset to assess model performance.

Accuracy evaluation can be performed using the genie-app tool. After computing the accuracy metrics,
the results will be returned as a JSON object containing the perplexity value.

Note

The accuracy computation resets the dialog and associated engine. It is recommended to run accuracy evaluation on a newly created dialog rather than in between dialog queries.

## Example on how to use Accuracy APIs with Dialog

The following example demonstrates how to integrate GenieAccuracy APIs with an existing dialog to compute perplexity metrics.

// Create Dialog Config
    GenieDialogConfig_Handle_t dialogConfigHandle = NULL;
    GenieDialogConfig_createFromJson(dialogConfigStr, &dialogConfigHandle);
    
    // Create Dialog
    GenieDialog_Handle_t dialogHandle = NULL;
    GenieDialog_create(dialogConfigHandle, &dialogHandle);
    
    // Create Accuracy Config
    GenieAccuracyConfig_Handle_t accuracyConfigHandle = NULL;
    GenieAccuracyConfig_createFromJson(accuracyConfigStr, &accuracyConfigHandle);
    
    // Create Accuracy Handle from Dialog
    GenieAccuracy_Handle_t accuracyHandle = NULL;
    GenieAccuracy_createFromDialog(accuracyConfigHandle, dialogHandle, &accuracyHandle);
    
    // Define allocation callback for accuracy results
    auto allocCallback = [](uint32_t size, void** buffer) -> Genie_Status_t {
        *buffer = reinterpret_cast<char*>(malloc(size));
        return (*buffer != nullptr) ? GENIE_STATUS_SUCCESS : GENIE_STATUS_ERROR_MEM_ALLOC;
    };
    
    // Compute accuracy metrics
    const char* jsonData = nullptr;
    Genie_Status_t status = GenieAccuracy_compute(accuracyHandle, allocCallback, &jsonData);
    
    if (status == GENIE_STATUS_SUCCESS && jsonData != nullptr) {
        // Process the accuracy results (JSON format)
        // Example output: {"perplexity": 5.234}
        std::cout << "Accuracy Results: " << jsonData << std::endl;
    }
    
    // Free the allocated memory for JSON data
    if (jsonData) {
        free(const_cast<char*>(jsonData));
    }
    
    // Free Accuracy Handle
    GenieAccuracy_free(accuracyHandle);
    
    // Free Accuracy Config
    GenieAccuracyConfig_free(accuracyConfigHandle);
    
    // Free Dialog
    GenieDialog_free(dialogHandle);
    
    // Free Dialog Config
    GenieDialogConfig_free(dialogConfigHandle);
    Copy to clipboard

## Example on how to use Accuracy APIs with Node

The following example demonstrates how to integrate GenieAccuracy APIs with a node (TextGenerator) to compute perplexity metrics.

// Create Node Config
    GenieNodeConfig_Handle_t nodeConfigHandle = NULL;
    GenieNodeConfig_createFromJson(nodeConfigStr, &nodeConfigHandle);
    
    // Create Node (TextGenerator)
    GenieNode_Handle_t nodeHandle = NULL;
    GenieNode_create(nodeConfigHandle, &nodeHandle);
    
    // Create Accuracy Config (can be NULL for default perplexity calculation)
    GenieAccuracyConfig_Handle_t accuracyConfigHandle = NULL;
    GenieAccuracyConfig_createFromJson(accuracyConfigStr, &accuracyConfigHandle);
    
    // Create Accuracy Handle from Node
    GenieAccuracy_Handle_t accuracyHandle = NULL;
    GenieAccuracy_createFromNode(accuracyConfigHandle, nodeHandle, &accuracyHandle);
    
    // Define allocation callback for accuracy results
    auto allocCallback = [](uint32_t size, void** buffer) -> Genie_Status_t {
        *buffer = reinterpret_cast<char*>(malloc(size));
        return (*buffer != nullptr) ? GENIE_STATUS_SUCCESS : GENIE_STATUS_ERROR_MEM_ALLOC;
    };
    
    // Compute accuracy metrics
    const char* jsonData = nullptr;
    Genie_Status_t status = GenieAccuracy_compute(accuracyHandle, allocCallback, &jsonData);
    
    if (status == GENIE_STATUS_SUCCESS && jsonData != nullptr) {
        // Process the accuracy results (JSON format)
        std::cout << "Accuracy Results: " << jsonData << std::endl;
    
        // Free the allocated memory for JSON data
        free(const_cast<char*>(jsonData));
    }
    
    // Free Accuracy Handle
    GenieAccuracy_free(accuracyHandle);
    
    // Free Accuracy Config (if created)
    if (accuracyConfigHandle != NULL) {
        GenieAccuracyConfig_free(accuracyConfigHandle);
    }
    
    // Free Node
    GenieNode_free(nodeHandle);
    
    // Free Node Config
    GenieNodeConfig_free(nodeConfigHandle);
    Copy to clipboard

## Example genie-app script

The following is an example script for utilizing accuracy features with genie-app
    
    # Version
    version
    
    # Dialog creation
    dialog config create config1 dialogConfig.json
    dialog create dialog1 config1
    
    # Accuracy Handle creation
    accuracy config create accConfig accConfig.json
    accuracy create accHandle1 accConfig dialog dialog1
    accuracy compute accHandle1 OUTPUT_FILE.json
    
    # Clean up
    accuracy config free accConfig
    accuracy free accHandle1
    dialog config free config1
    dialog free dialog1
    ls
    
    # Exit
    exit
    Copy to clipboard

## Accuracy Configuration

The accuracy configuration JSON should specify the dataset path and the type of accuracy metric to compute.
For more details on the accuracy configuration format, please refer to the [Genie Accuracy JSON Configuration](https://docs.qualcomm.com/doc/80-63442-10/topic/json.html#genie-accuracy-json-config-string) documentation.

Last Published: Aug 06, 2026

[Previous Topic
facebook/wt19-en-de](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/wt19-en-de.md) [Next Topic
Genie DLC](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/dlc.md)