| 1441 |
ariadna |
1 |
@report @core_ai
|
|
|
2 |
Feature: AI usage report displays recorded AI data
|
|
|
3 |
In order to view AI usage data
|
|
|
4 |
As a manager or admin
|
|
|
5 |
I can access the AI usage report
|
|
|
6 |
|
|
|
7 |
Background:
|
|
|
8 |
Given the following "users" exist:
|
|
|
9 |
| username | firstname | lastname | email |
|
|
|
10 |
| manager1 | Manager | One | manager1@example.com |
|
|
|
11 |
| student1 | Student | One | student1@example.com |
|
|
|
12 |
| student2 | Student | Two | student2@example.com |
|
|
|
13 |
And the following "role assigns" exist:
|
|
|
14 |
| user | role | contextlevel | reference |
|
|
|
15 |
| manager1 | manager | System | |
|
|
|
16 |
And the following "core_ai > ai providers" exist:
|
|
|
17 |
| provider | name | enabled | apikey | orgid |
|
|
|
18 |
| aiprovider_openai | OpenAI API test | 1 | 123 | abc |
|
|
|
19 |
And the following "core_ai > ai actions" exist:
|
|
|
20 |
| actionname | user | success | provider | contextid | prompttokens | completiontokens |
|
|
|
21 |
| generate_text | student1 | 1 | aiprovider_openai | 1 | 22 | 33 |
|
|
|
22 |
| summarise_text | student1 | 0 | aiprovider_openai | 1 | | |
|
|
|
23 |
| explain_text | student1 | 1 | aiprovider_openai | 1 | 44 | 55 |
|
|
|
24 |
| generate_image | student2 | 1 | aiprovider_azureai | 1 | | |
|
|
|
25 |
|
|
|
26 |
Scenario: Managers can view the AI usage report
|
|
|
27 |
Given I am logged in as "manager1"
|
|
|
28 |
When I navigate to "Reports > AI reports > AI usage" in site administration
|
|
|
29 |
Then the following should exist in the "AI usage" table:
|
|
|
30 |
| Action | First name | Provider | Success | Prompt tokens | Completion tokens |
|
|
|
31 |
| Generate text | Student One | OpenAI API provider | Yes | 22 | 33 |
|
|
|
32 |
| Summarise text | Student One | OpenAI API provider | No | | |
|
|
|
33 |
| Explain text | Student One | OpenAI API provider | Yes | 44 | 55 |
|
|
|
34 |
| Generate image | Student Two | Azure AI API provider | Yes | | |
|
|
|
35 |
|
|
|
36 |
@javascript
|
|
|
37 |
Scenario: Managers can filter the AI usage report
|
|
|
38 |
Given I am logged in as "manager1"
|
|
|
39 |
When I navigate to "Reports > AI reports > AI usage" in site administration
|
|
|
40 |
And I click on "Filters" "button"
|
|
|
41 |
And I set the field "Provider value" in the "Provider" "core_reportbuilder > Filter" to "OpenAI API provider"
|
|
|
42 |
And I click on "Apply" "button" in the "[data-region='report-filters']" "css_element"
|
|
|
43 |
Then the following should exist in the "AI usage" table:
|
|
|
44 |
| Action | First name | Provider | Success | Prompt tokens | Completion tokens |
|
|
|
45 |
| Generate text | Student One | OpenAI API provider | Yes | 22 | 33 |
|
|
|
46 |
| Summarise text | Student One | OpenAI API provider | No | | |
|
|
|
47 |
| Explain text | Student One | OpenAI API provider | Yes | 44 | 55 |
|
|
|
48 |
And I should not see "Azure AI API provider" in the "AI usage" "table"
|
|
|
49 |
And I set the following fields in the "Action" "core_reportbuilder > Filter" to these values:
|
|
|
50 |
| Action operator | Is equal to |
|
|
|
51 |
| Action value | Generate text |
|
|
|
52 |
And I click on "Apply" "button" in the "[data-region='report-filters']" "css_element"
|
|
|
53 |
And the following should exist in the "AI usage" table:
|
|
|
54 |
| Action | First name | Provider | Success | Prompt tokens | Completion tokens |
|
|
|
55 |
| Generate text | Student One | OpenAI API provider | Yes | 22 | 33 |
|
|
|
56 |
And I should not see "Summarise text" in the "AI usage" "table"
|
|
|
57 |
And I should not see "Azure AI API Provider" in the "AI usage" "table"
|