AWS Certified AI Practitioner Practice Exam

Which factor will drive the inference costs when using Amazon Bedrock for large language models (LLM)?

1. Number of tokens consumed

2. Temperature value

3. Amount of data used to train the LLM

4. Total training time

The correct answer is: Number of tokens consumed

The factor that drives inference costs when using Amazon Bedrock for large language models (LLMs) is the number of tokens consumed. Inference in language models involves the processing of input text, which is measured in tokens. Each time a model is queried, the length of the input (in tokens) directly impacts the compute resources required for processing the request and generating the output. A higher token count means that the model will have to perform more operations, resulting in increased costs. The temperature value, while influential in terms of the creativity and variability of the generated responses, does not itself affect inference costs. It's primarily a parameter used during inference to control randomness in the output rather than a cost driving factor. Regarding the amount of data used to train the LLM and total training time, these factors are relevant to the model training phase rather than inference. They influence initial model training costs but do not impact the costs incurred during the inference phase when the model is being used to generate responses to queries. Therefore, the correct answer directly correlates to how token consumption affects the computing resources used during the inference process.