What is a User Group?
A User Group organizes users, metrics, and products within an organization to streamline human evaluation workflows. User Groups serve three key purposes:- Assigning evaluators: Group users who should review and annotate AI outputs for specific metrics.
- Scoping metrics: Link metrics to groups so that only users in those groups see pending evaluations for those metrics.
- Linking products: Link products to a group, or mark a group as linked to all products of the organization, current and future. A MEMBER reaches the union of the products that all its groups link. When one of its groups links all products, it reaches every product, including a product that no other group links. A MEMBER in no group sees no product. A MEMBER creates and edits only inside those products, and creates a product only when one of its groups links all products and its list allows
product.create(a group that follows Editor does). A MEMBER sees every metric of the organization when its lists allowmetric.createormetric.update, or when one of its groups links all products and allowsproduct.read. Otherwise it sees only the metrics that its products evaluate, link to a specification, or monitor. It edits them when its lists allowmetric.update(see Permission lists). OWNER and ADMIN users see every product and every metric of the organization, whatever their groups.
Permission lists
An OWNER or ADMIN can give a group a permission list: the actions its users may do on the products the group links, and on metrics, models and comments across the organization. A permission is one action on one entity, writtenentity.action, for example product.read or analytics.readCoverage. A group with a list allows only the actions on it. Permission list ids lists every id a list may hold.
- The role caps the list. A MEMBER may do an action on a product when their role allows it and at least one of their groups that reaches the product lists it. A list entry above the role does nothing. A list never limits an OWNER or ADMIN. For example, a group with the Reader preset never lets a MEMBER write.
- Groups add up. A user in two groups on one product may do an action when either group lists it. A list never removes an action that another group allows.
- Editor means every action. A group that follows the Editor preset allows every action its members’ role allows, including actions added later. An empty list grants nothing. A user in no group reaches no product, and product lists answer 200 with an empty page.
- Metrics, models and comments are shared across products. A
metric,modelorcommententry is not limited to the products the group links:metric.updateedits any metric of the organization. What See shows can still follow the products, as What See shows says. A group with no products can hold these entries, for example a Metric authors group withmetric.read,metric.createandmetric.update. A user in no group gets no metric, model or comment action.comment.deletedeletes only the user’s own comments. Creating a revision of a metric needs bothmetric.createandmetric.update. - A list needs reads. A list that holds an entry on a product entity holds
product.read, and an action on an entity needs thereadof that entity. Some actions also need a read of another entity: an action that uses metrics (evaluations, runs, specifications, monitors, analytics) needsmetric.read, a version action needsmodel.read,comment.createandevaluation.readInferenceneedinferenceResult.read, and reading traces, sessions or inference results needscomment.read. The API answers 400 otherwise, and names the missing read. It also answers 400 for an entry that no list can grant, such asuserGroup.update. - An all-products group that follows Editor still applies. A member who is also in one keeps every action their role allows. A narrower group binds that user only after an OWNER removes them from the all-products group, or gives it another preset or a list of its own.
- A list without a read refuses some lists. When every group of a user leaves out
test.read,GET /testsanswers 403. The same holds formonitor.readandGET /monitors,metric.readandGET /metrics, andmodel.readandGET /models, so a user in no group gets 403 fromGET /metricsandGET /models, and an empty page fromGET /testsandGET /monitors. Other lists, such asGET /sessionsandGET /runs, answer 200 with an empty page. - Some actions stay with the role. Only actions on entities that belong to a product, and on metrics, models and comments, come from lists. Every other action stays with the role. Datasets with no product (Galtea-wide datasets) stay readable by id whatever the list says.
GET /testslists one only whenidsortestCaseIdsnames it.product.softDeletein a list means the user’s own products only. Creating a product needs an all-products group whose list allowsproduct.create. - Analytics follows
analytics.read. Withoutanalytics.readon a product, a route for that one product answers 403. The dashboard hides the product’s Analytics section and opens the first section the user may see. The two routes for many products keep the product:GET /analytics/summaryanswers an empty row for it, andGET /analytics/productsanswers its row with no stats. The summary boxes of the organization products page count only the products withanalytics.read, and answer 403 when the request names another. Withoutanalytics.readCoverage, the dashboard hides the Coverage section. - The insights narrative summarizes evaluation reasons. It is written from the reasons the judge gave in evaluations. A group with
analytics.readbut withoutevaluation.readcan still read text drawn from those reasons.
What See shows
For example, a Support group that links one product and lists
metric.read sees only the metrics that product uses. The dashboard’s Reaches column shows the same answer for the boxes ticked now.
A Business group is a typical example. List product.read, version.read, specification.read, analytics.read, analytics.readCoverage, metric.read and model.read, and link the products. Its users see the analytics views of those products, including the version filter and the insights, which read the versions and specifications. They see no tests, sessions or evaluations: the dataset and monitor lists answer 403, and the other lists are empty.
A list change takes effect on the next API request. The dashboard shows it when the user returns to the tab or reloads. The API decides: a dashboard control can still show an action that the API then refuses with 403. In a frozen organization, a custom list never grants a write: the freeze keeps users read-only.
Presets
A preset is a permission list that Galtea maintains: Editor, Human evaluator or Reader. Reader holds the plain read of each entity, such asproduct.read and session.read. It also holds evaluation.readInference, so its members see cost and latency. It leaves out the other reads: analytics.readCoverage, evaluation.readMetadata, product.readConfiguration, product.readTutorial, test.readSection and version.readSection. Human evaluator adds to Reader what a human evaluator may do on evaluations. Editor is the default, and allows every action the role allows. A group either follows a preset or holds its own list:
- Follows a preset. The group stores only the preset. When Galtea changes what the preset grants, for example to add the permissions of a new feature, the group changes with it. Its owner never updates it by hand. The dashboard names it, for example “Reader permissions”.
- Holds its own list. The group stores its permissions, and a change to a preset changes nothing. The dashboard shows its counts, for example “Reads 5, Writes 2, Deletes 1”.
POST /userGroups creates the group, or later with PATCH /userGroups/{id} or user_groups.update:
permissionPresetset toEDITOR,HUMAN_EVALUATORorREADERmakes the group follow that preset.permissionPreset: nullwith apermissionslist makes the group hold that list.- A
permissionslist alone also makes the group hold it.permissions: null, or no field onPOST, makes the group follow Editor. Leaving both out ofPATCHkeeps the permissions. - A preset together with a list, or
permissionPreset: nullwith no list, answers 400.
POST, productIds links products to the new group. A list that breaks a rule, or a product that does not exist or is in another organization, answers 400 and creates no group. Both endpoints answer 400 for a body key they do not know. GET /userGroups/grantablePermissions lists every permission a list may hold, with a short description of each and the reads it requires, and the presets, each with its description and its permissions today. The default preset has isDefault: true.
Managing User Groups
Only an OWNER or ADMIN may create, edit, or delete a user group, or change its users or products. A MEMBER can read every group of the organization. A MEMBER can also link or unlink a group of the organization to a metric that the MEMBER may edit. User groups can be managed in two places:- Dashboard: Navigate to an organization’s Groups tab to create, edit, and delete groups. The create and edit forms set the group’s products, or All products. In a group’s Users, Metrics and Products tabs, an OWNER or ADMIN links entries and unlinks the selected ones. In the Products tab, the row menu also unlinks one product. Unlinking never deletes a user, a metric or a product. Creating a user, a metric or a product from one of these tabs links it to the group: the new user joins the group, the new product is linked to it, and the metric form selects the group. A blank metric form starts as Human Evaluation. Only a Human Evaluation metric keeps the group. This needs permission to edit the group, and a new metric or product is linked only to a group of your own organization. The Metrics tab links only Human Evaluation metrics. The platform refuses to unlink a metric that has evaluations in the group’s organization, or a Human Evaluation metric from its last group. In the Products tab, while All products is on, the tab shows only a note. The group keeps its stored links, and they apply again when All products is turned off. The edit form shows them. In the create form, turning on All products clears the products that you picked. From the user detail page, an OWNER or ADMIN can assign and unassign groups. From the metric detail page, any user who may edit the metric can assign and unassign groups.
- Invite form: When an OWNER or ADMIN invites a MEMBER, the form asks for their groups. Each group shows its products, users and permissions, and a click on one opens a dialog with the details. A summary beside the form shows the permissions the user gets on each product. When the organization has no group yet, a dialog after the save of a MEMBER offers to create the first one. The new group holds the user, unless the invited address already had an account.
- Invite an OWNER: The form hides the groups for an OWNER, who reaches every product. An OWNER still needs a group to evaluate a Human Evaluation metric. So an OWNER invited from a group’s Users tab joins that group.
- Roles on the form: The role list offers OWNER and MEMBER, plus the staff roles to an ADMIN. For a read-only user or a human evaluator, invite a MEMBER and pick a group with the Reader or Human evaluator preset.
- Edit user page: The edit user page has the same Groups section. When a role change makes a user a MEMBER while the organization has no group, the same dialog opens after the save.
- Users tab: The organization’s Users tab shows each user’s role and their groups with their permissions. A click on a group opens its products, users and permissions. It flags a MEMBER with no group.
- SDK: Use the
galtea.user_groupsservice for programmatic management. See the Human Evaluation tutorial for a step-by-step guide.
SDK Integration
User Group SDK API
Create, list, update, and delete user groups programmatically. Link and unlink users, metrics, and products.
User Group Properties
IDs of metrics linked to this group. A MEMBER gets only the IDs of the metrics it can see. A MEMBER sees all of them when its lists allow
metric.create or metric.update, or when one of its groups links all products and allows product.read.IDs of products linked to this group. The products a group links are the only products its MEMBER users can reach. A MEMBER gets only the IDs of the products it can see, in every group.
When
true, the group is linked to every product of the organization, current and future,
regardless of product_ids. A MEMBER in the group then reaches every product.The preset the group follows:
EDITOR, HUMAN_EVALUATOR or READER. None means the group holds its own list.What the group grants, sorted: the list of the preset it follows, or its own list.
None means the group follows Editor and allows every action the role of its members allows. An empty list grants nothing.Related
Concepts overview
How Galtea’s concepts connect — diagram + per-entity quick reference.
Human Evaluation Tutorial
Step-by-step guide to the full annotation workflow with screenshots.
User Group SDK API
Programmatic management of user groups.