google-native.dialogflow/v2.getEvaluation

func LookupEvaluation(ctx *Context, args *LookupEvaluationArgs, opts ...InvokeOption) (*LookupEvaluationResult, error)
func LookupEvaluationOutput(ctx *Context, args *LookupEvaluationOutputArgs, opts ...InvokeOption) LookupEvaluationResultOutput

> Note: This function is named LookupEvaluation in the Go SDK.

ConversationModelId This property is required. string

EvaluationId This property is required. string

Location This property is required. string

Project string

ConversationModelId This property is required. string

EvaluationId This property is required. string

Location This property is required. string

Project string

conversationModelId This property is required. String

evaluationId This property is required. String

location This property is required. String

project String

conversation_model_id This property is required. str

evaluation_id This property is required. str

location This property is required. str

project str

conversationModelId This property is required. String

evaluationId This property is required. String

location This property is required. String

project String

CreateTime string

Creation time of this model.

DisplayName string

Optional. The display name of the model evaluation. At most 64 bytes long.

EvaluationConfig Pulumi.GoogleNative.Dialogflow.V2.Outputs.GoogleCloudDialogflowV2EvaluationConfigResponse

Optional. The configuration of the evaluation task.

Name string

The resource name of the evaluation. Format: projects//conversationModels//evaluations/

RawHumanEvalTemplateCsv string

Human eval template in csv format. It tooks real-world conversations provided through input dataset, generates example suggestions for customer to verify quality of the model. For Smart Reply, the generated csv file contains columns of Context, (Suggestions,Q1,Q2)*3, Actual reply. Context contains at most 10 latest messages in the conversation prior to the current suggestion. Q1: "Would you send it as the next message of agent?" Evaluated based on whether the suggest is appropriate to be sent by agent in current context. Q2: "Does the suggestion move the conversation closer to resolution?" Evaluated based on whether the suggestion provide solutions, or answers customer's question or collect information from customer to resolve the customer's issue. Actual reply column contains the actual agent reply sent in the context.

SmartReplyMetrics Pulumi.GoogleNative.Dialogflow.V2.Outputs.GoogleCloudDialogflowV2SmartReplyMetricsResponse

Only available when model is for smart reply.

CreateTime string

Creation time of this model.

DisplayName string

Optional. The display name of the model evaluation. At most 64 bytes long.

EvaluationConfig GoogleCloudDialogflowV2EvaluationConfigResponse

Optional. The configuration of the evaluation task.

Name string

The resource name of the evaluation. Format: projects//conversationModels//evaluations/

RawHumanEvalTemplateCsv string

Human eval template in csv format. It tooks real-world conversations provided through input dataset, generates example suggestions for customer to verify quality of the model. For Smart Reply, the generated csv file contains columns of Context, (Suggestions,Q1,Q2)*3, Actual reply. Context contains at most 10 latest messages in the conversation prior to the current suggestion. Q1: "Would you send it as the next message of agent?" Evaluated based on whether the suggest is appropriate to be sent by agent in current context. Q2: "Does the suggestion move the conversation closer to resolution?" Evaluated based on whether the suggestion provide solutions, or answers customer's question or collect information from customer to resolve the customer's issue. Actual reply column contains the actual agent reply sent in the context.

SmartReplyMetrics GoogleCloudDialogflowV2SmartReplyMetricsResponse

Only available when model is for smart reply.

createTime String

Creation time of this model.

displayName String

Optional. The display name of the model evaluation. At most 64 bytes long.

evaluationConfig GoogleCloudDialogflowV2EvaluationConfigResponse

Optional. The configuration of the evaluation task.

name String

The resource name of the evaluation. Format: projects//conversationModels//evaluations/

rawHumanEvalTemplateCsv String

Human eval template in csv format. It tooks real-world conversations provided through input dataset, generates example suggestions for customer to verify quality of the model. For Smart Reply, the generated csv file contains columns of Context, (Suggestions,Q1,Q2)*3, Actual reply. Context contains at most 10 latest messages in the conversation prior to the current suggestion. Q1: "Would you send it as the next message of agent?" Evaluated based on whether the suggest is appropriate to be sent by agent in current context. Q2: "Does the suggestion move the conversation closer to resolution?" Evaluated based on whether the suggestion provide solutions, or answers customer's question or collect information from customer to resolve the customer's issue. Actual reply column contains the actual agent reply sent in the context.

smartReplyMetrics GoogleCloudDialogflowV2SmartReplyMetricsResponse

Only available when model is for smart reply.

create_time str

Creation time of this model.

display_name str

Optional. The display name of the model evaluation. At most 64 bytes long.

evaluation_config GoogleCloudDialogflowV2EvaluationConfigResponse

Optional. The configuration of the evaluation task.

name str

The resource name of the evaluation. Format: projects//conversationModels//evaluations/

raw_human_eval_template_csv str

Human eval template in csv format. It tooks real-world conversations provided through input dataset, generates example suggestions for customer to verify quality of the model. For Smart Reply, the generated csv file contains columns of Context, (Suggestions,Q1,Q2)*3, Actual reply. Context contains at most 10 latest messages in the conversation prior to the current suggestion. Q1: "Would you send it as the next message of agent?" Evaluated based on whether the suggest is appropriate to be sent by agent in current context. Q2: "Does the suggestion move the conversation closer to resolution?" Evaluated based on whether the suggestion provide solutions, or answers customer's question or collect information from customer to resolve the customer's issue. Actual reply column contains the actual agent reply sent in the context.

smart_reply_metrics GoogleCloudDialogflowV2SmartReplyMetricsResponse

Only available when model is for smart reply.

createTime String

Creation time of this model.

displayName String

Optional. The display name of the model evaluation. At most 64 bytes long.

evaluationConfig Property Map

Optional. The configuration of the evaluation task.

name String

The resource name of the evaluation. Format: projects//conversationModels//evaluations/

rawHumanEvalTemplateCsv String

Human eval template in csv format. It tooks real-world conversations provided through input dataset, generates example suggestions for customer to verify quality of the model. For Smart Reply, the generated csv file contains columns of Context, (Suggestions,Q1,Q2)*3, Actual reply. Context contains at most 10 latest messages in the conversation prior to the current suggestion. Q1: "Would you send it as the next message of agent?" Evaluated based on whether the suggest is appropriate to be sent by agent in current context. Q2: "Does the suggestion move the conversation closer to resolution?" Evaluated based on whether the suggestion provide solutions, or answers customer's question or collect information from customer to resolve the customer's issue. Actual reply column contains the actual agent reply sent in the context.

smartReplyMetrics Property Map

Only available when model is for smart reply.

Datasets This property is required. List<Pulumi.GoogleNative.Dialogflow.V2.Inputs.GoogleCloudDialogflowV2InputDatasetResponse>

Datasets used for evaluation.

SmartComposeConfig This property is required. Pulumi.GoogleNative.Dialogflow.V2.Inputs.GoogleCloudDialogflowV2EvaluationConfigSmartComposeConfigResponse

Configuration for smart compose model evalution.

SmartReplyConfig This property is required. Pulumi.GoogleNative.Dialogflow.V2.Inputs.GoogleCloudDialogflowV2EvaluationConfigSmartReplyConfigResponse

Configuration for smart reply model evalution.

Datasets This property is required. []GoogleCloudDialogflowV2InputDatasetResponse

Datasets used for evaluation.

SmartComposeConfig This property is required. GoogleCloudDialogflowV2EvaluationConfigSmartComposeConfigResponse

Configuration for smart compose model evalution.

SmartReplyConfig This property is required. GoogleCloudDialogflowV2EvaluationConfigSmartReplyConfigResponse

Configuration for smart reply model evalution.

datasets This property is required. List<GoogleCloudDialogflowV2InputDatasetResponse>

Datasets used for evaluation.

smartComposeConfig This property is required. GoogleCloudDialogflowV2EvaluationConfigSmartComposeConfigResponse

Configuration for smart compose model evalution.

smartReplyConfig This property is required. GoogleCloudDialogflowV2EvaluationConfigSmartReplyConfigResponse

Configuration for smart reply model evalution.

datasets This property is required. Sequence[GoogleCloudDialogflowV2InputDatasetResponse]

Datasets used for evaluation.

smart_compose_config This property is required. GoogleCloudDialogflowV2EvaluationConfigSmartComposeConfigResponse

Configuration for smart compose model evalution.

smart_reply_config This property is required. GoogleCloudDialogflowV2EvaluationConfigSmartReplyConfigResponse

Configuration for smart reply model evalution.

datasets This property is required. List<Property Map>

Datasets used for evaluation.

smartComposeConfig This property is required. Property Map

Configuration for smart compose model evalution.

smartReplyConfig This property is required. Property Map

Configuration for smart reply model evalution.

AllowlistDocument This property is required. string

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart compose model.

MaxResultCount This property is required. int

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

AllowlistDocument This property is required. string

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart compose model.

MaxResultCount This property is required. int

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

allowlistDocument This property is required. String

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart compose model.

maxResultCount This property is required. Integer

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

allowlist_document This property is required. str

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart compose model.

max_result_count This property is required. int

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

allowlistDocument This property is required. String

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart compose model.

maxResultCount This property is required. Number

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

AllowlistDocument This property is required. string

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart reply model.

MaxResultCount This property is required. int

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

AllowlistDocument This property is required. string

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart reply model.

MaxResultCount This property is required. int

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

allowlistDocument This property is required. String

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart reply model.

maxResultCount This property is required. Integer

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

allowlist_document This property is required. str

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart reply model.

max_result_count This property is required. int

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

allowlistDocument This property is required. String

The allowlist document resource name. Format: projects//knowledgeBases//documents/. Only used for smart reply model.

maxResultCount This property is required. Number

The model to be evaluated can return multiple results with confidence score on each query. These results will be sorted by the descending order of the scores and we only keep the first max_result_count results as the final results to evaluate.

Dataset This property is required. string

ConversationDataset resource name. Format: projects//locations//conversationDatasets/

Dataset This property is required. string

ConversationDataset resource name. Format: projects//locations//conversationDatasets/

dataset This property is required. String

ConversationDataset resource name. Format: projects//locations//conversationDatasets/

dataset This property is required. str

ConversationDataset resource name. Format: projects//locations//conversationDatasets/

dataset This property is required. String

ConversationDataset resource name. Format: projects//locations//conversationDatasets/

AllowlistCoverage This property is required. double

Percentage of target participant messages in the evaluation dataset for which similar messages have appeared at least once in the allowlist. Should be [0, 1].

ConversationCount This property is required. string

Total number of conversations used to generate this metric.

TopNMetrics This property is required. List<Pulumi.GoogleNative.Dialogflow.V2.Inputs.GoogleCloudDialogflowV2SmartReplyMetricsTopNMetricsResponse>

Metrics of top n smart replies, sorted by TopNMetric.n.

AllowlistCoverage This property is required. float64

Percentage of target participant messages in the evaluation dataset for which similar messages have appeared at least once in the allowlist. Should be [0, 1].

ConversationCount This property is required. string

Total number of conversations used to generate this metric.

TopNMetrics This property is required. []GoogleCloudDialogflowV2SmartReplyMetricsTopNMetricsResponse

Metrics of top n smart replies, sorted by TopNMetric.n.

allowlistCoverage This property is required. Double

Percentage of target participant messages in the evaluation dataset for which similar messages have appeared at least once in the allowlist. Should be [0, 1].

conversationCount This property is required. String

Total number of conversations used to generate this metric.

topNMetrics This property is required. List<GoogleCloudDialogflowV2SmartReplyMetricsTopNMetricsResponse>

Metrics of top n smart replies, sorted by TopNMetric.n.

allowlist_coverage This property is required. float

Percentage of target participant messages in the evaluation dataset for which similar messages have appeared at least once in the allowlist. Should be [0, 1].

conversation_count This property is required. str

Total number of conversations used to generate this metric.

top_n_metrics This property is required. Sequence[GoogleCloudDialogflowV2SmartReplyMetricsTopNMetricsResponse]

Metrics of top n smart replies, sorted by TopNMetric.n.

allowlistCoverage This property is required. Number

Percentage of target participant messages in the evaluation dataset for which similar messages have appeared at least once in the allowlist. Should be [0, 1].

conversationCount This property is required. String

Total number of conversations used to generate this metric.

topNMetrics This property is required. List<Property Map>

Metrics of top n smart replies, sorted by TopNMetric.n.

N This property is required. int

Number of retrieved smart replies. For example, when n is 3, this evaluation contains metrics for when Dialogflow retrieves 3 smart replies with the model.

Recall This property is required. double

Defined as number of queries whose top n smart replies have at least one similar (token match similarity above the defined threshold) reply as the real reply divided by number of queries with at least one smart reply. Value ranges from 0.0 to 1.0 inclusive.

N This property is required. int

Number of retrieved smart replies. For example, when n is 3, this evaluation contains metrics for when Dialogflow retrieves 3 smart replies with the model.

Recall This property is required. float64

Defined as number of queries whose top n smart replies have at least one similar (token match similarity above the defined threshold) reply as the real reply divided by number of queries with at least one smart reply. Value ranges from 0.0 to 1.0 inclusive.

n This property is required. Integer

Number of retrieved smart replies. For example, when n is 3, this evaluation contains metrics for when Dialogflow retrieves 3 smart replies with the model.

recall This property is required. Double

Defined as number of queries whose top n smart replies have at least one similar (token match similarity above the defined threshold) reply as the real reply divided by number of queries with at least one smart reply. Value ranges from 0.0 to 1.0 inclusive.

n This property is required. int

Number of retrieved smart replies. For example, when n is 3, this evaluation contains metrics for when Dialogflow retrieves 3 smart replies with the model.

recall This property is required. float

Defined as number of queries whose top n smart replies have at least one similar (token match similarity above the defined threshold) reply as the real reply divided by number of queries with at least one smart reply. Value ranges from 0.0 to 1.0 inclusive.

n This property is required. Number

Number of retrieved smart replies. For example, when n is 3, this evaluation contains metrics for when Dialogflow retrieves 3 smart replies with the model.

recall This property is required. Number

Defined as number of queries whose top n smart replies have at least one similar (token match similarity above the defined threshold) reply as the real reply divided by number of queries with at least one smart reply. Value ranges from 0.0 to 1.0 inclusive.