IDEIA

Árvore de páginas

Este documento compartilha o Release Notes relacionado à plataforma Carol - 3.18 e 3.19 (incluindo os deployment semanais).


Carol Platform

Carol App

Carol App installation faster (https://trello.com/c/JLX4Vxf6)

  • It was applied improvements on the process that installs a Carol App - making it faster.

Carol App: Autoscale feature (https://trello.com/c/JhUoZJ96)

  • Carol App has the support to autoscale an Online Carol App. If you want to see details, please check out the documentation: https://docs.carol.ai/docs/manifest-file#understanding-the-manifest-file

Carol App: custom health check endpoint (https://trello.com/c/ze01dkjk)

  • Carol App has support to a custom URL for the health check. The health check URL is important to Carol to determine if the application is running correctly. Some applications (most of them deploy Web + BE resources together) can't follow the standard implementation of the health check (return 200 on GET to /). Also, now it is possible to customize the port for the Online Carol App (default is 5000). More details here: https://docs.carol.ai/docs/manifest-file#understanding-the-manifest-file

Carol App not showing the schedule and last run for Carol App Processes (https://trello.com/c/7RjMqbaG)

  • It is fixed a situation that some environments were not showing the schedule and last run for Carol App Process. No action needed in environments.

Carol Data Storage

Carol App is propagating the schedule for consolidation (staging tables and data models) (https://trello.com/c/SZsZzjil)

  • The consolidation task rebuilds the data (for staging tables and data models) and optimizes the data distributing it accordingly on the data files. It also removes duplicated records (when it was updated) or deleted records. The Carol App is propagating the schedules to all environments with the Carol App installed.

Connector

Staging table: manage fields and staging sample data (https://trello.com/c/P5VjnRCo)

  • Applied some fixes in the flow for the staging table, making some features more visible. The manage field flow is now accessible externally the View Sample Data link. When the staging table is lookup, it will have by default 100% of sample data, and the link to see staging data is called View Data.

Data Access Level

Fixed some cases on Data Access Level flow (https://trello.com/c/cj4ktW7p)

  • The flow to manage data access level got some fixes, and now it is showing the right number of users when the user applies a filter to add more users to the group.

Data Access Level: bug fix when the environment has no group configured (https://trello.com/c/vtmcvuTq)

  • Fixed some bugs on Data Access Level flow, for example, when creating a data access level in an environment with none.

Data Model

Data Model and projects (https://trello.com/c/5HHMAbZf)

  • It is fixed a bug that was showing the same project several times.

Data Processing

Publishing and Playing a staging table not reprocessing the whole dataset (https://trello.com/c/sa04dIrT)

  • We applied two critical changes in the Staging Data processing. When starting the data processing, it is no more triggering the data processing for the entire dataset (before it reprocesses all data in CDS). When publishing a mapping, the user can publish it without processing the whole dataset - attention that before it was automatically reprocessing the whole dataset.

The API to delete staging records propagating to golden records got fixed (https://trello.com/c/i4zLimLG)

  • The API got a fix where some cases did not propagate the deletion to golden records, resulting in the golden records related to the staging tables not being removed.

The API to delete values from the staging table got fixed (https://trello.com/c/ZB0CTQ2X)

  • It got a fix for a case where the crosswalk has long values. This bug was impacting not all cases.

Bugfix on Mapping and Cleanse rule to send data to staging (https://trello.com/c/6J2gYzgB)

  • The mapping and cleanse rule to send data to the staging area is fixed.

Carol Platform applied optimization when processing ETL with output fields defined (https://trello.com/c/qiidIYE7)

  • Carol applies optimization when the ETL has defined output fields. Other than optimizing the data processing, it can help the environment saving storage.

Clean and Reprocess on staging area warning the user (https://trello.com/c/omPpPBaI)

  • The process clean and reprocess on the staging table has a step to warm the user about the effects of this action. When the user clean and reprocess the data, all golden records coming from the specified staging table will be deleted, and the staging data will be processed again. In case the golden records have data coming from different staging tables, other staging tables must be reprocessed (without the clean option).

Improved integration between Tasks and internal data processing tasks (https://trello.com/c/YMf7NfPS)

  • It was improved some cases where Carol was not updating the task with the real status when processing data (staging to golden record). Now it is working correctly.

Reviewed the merge process for optimizations (https://trello.com/c/RNR4wn62)

  • The merge process was reviewed, and some optimizations applied to it.

Some records not being merges by staging table identifier (https://trello.com/c/eFIWhCGI)

  • Fixed a case that was causing staging records not being merged by the Staging Table Identifier (crosswalk).

Endpoint sync working properly with ETL (https://trello.com/c/OVfVoS5P)

  • The endpoint sync now works correctly when the first staging table has an ETL configuration (before it was inserting the records following the standard process to send data to Carol). This endpoint now processes the whole pipeline.

Environment

Defining the Environment Tag (https://trello.com/c/r4ficFW6)

  • The Organization Admin or the Environment Admin can set the environment's tag for the environment. The tag is limited to Development, QA, and Production. This tag has no impact on Carol.

ETL

ETL and output fields (https://trello.com/c/W8b9TqBP)

  • The user can define the fields he wants to output when defining an ETL rule. This flow allows us to optimize the performance for the data processing and better use the storage in Carol.

ETL is allowing to specify more conditions for split (https://trello.com/c/VMgXmAv9)

  • The Split ETL configuration allows specifying more than one condition. It allows us to make simpler rules when defining split ETLs.

Explore

Explore is allowing to reprocess rejected records when Evict is enabled. (https://trello.com/c/2DQzzl8o)

  • The module explore allows the reprocess of rejected records when the data model has the evict configuration enabled. The evict configuration creates golden records without the meta-information related to the staging records, but all rejected records have this meta-information.

Mapping and Cleanse Rule

Mapping and Cleanse: download snapshot when there is a draft version (https://trello.com/c/xzMQVHis)

  • Now it is showing the option to download the draft snapshot only when there is a draft snapshot available.

Organization

An endpoint to get the current organization (https://trello.com/c/iAqOsLOd)

  • There is an endpoint to get the logged user's organization: /v1/organizations/current. This endpoint works for authenticated and non-authenticated users.

Platform

Release Notes and Documentation available on TDN (https://trello.com/c/JZYZiDwF)

  • The documentation now is available in TOTVS TDN also (official repository for documentation). You can access it through the following URL: https://tdn.totvs.com/display/CARL/Integrando+com+a+Carol. The Release Notes is available under this platform: https://tdn.totvs.com/pages/viewpage.action?pageId=549512273

Improved memory management on Explore (https://trello.com/c/AdhXaAQ1)

  • It is fixed in some cases where the browser was getting slow as the user uses Carol.

Name limit for Organizations, Environments, and Carol Apps (https://trello.com/c/Bxyu8JcF)

  • Carol is enforcing a name size limit of Carol App name: 20 characters, Organization name: 25 characters, and Environment name: 40 characters.

Visual improvements in Data Model and Mapping (https://trello.com/c/n7Ngjqrz)

  • Sorting the relationships of a Data Model and the fields in mapping and cleanse rule alphabetically.

SQL

Bug fixes on SQL interface (https://trello.com/c/26T8Kl9U)

  • The SQL interface (path carol-ui/sql in an environment, for example: https://poffo.carol.ai/treinamento/carol-ui/sql) got some fixes, and it is much more stable now.

Delete data model's data allowing to delete SQL data (https://trello.com/c/0Gfm6gTT)

  • Now it is possible to delete the data in the SQL storage type when deleting the data from a data model. The SQL feature is available only for selected projects.

Tenant Management

Manage Tokens of the Environment (https://trello.com/c/zfCXzZE9)

  • Carol has a specific flow to manage (create and revoke) tokens related to the environment (known as Connector Token). This flow centralizes all actions on Environment Admin flow and removes it from the Connector level. If you want more details on how to use the Tokens, take a look at this documentation: https://tdn.totvs.com/pages/viewpage.action?pageId=552107176

User Management

Environment management on Organization Level allows adding users (https://trello.com/c/wss1Hk3f)

  • The organization admin can manage environments adding users to it, skipping the invitation flow. This feature is important when the environment has the TOTVS Identity enabled. Previously, it was released the flow to add environments to users.

User creation failing when user already exists on Organization level (https://trello.com/c/tNddjqcw)

  • Carol is better handling cases when the user already exists on Organization Level. Before it, the user creation flow was failing without creating the user in the environment level. Now the user is being created normally. This case was impacting user creation through the User Data Model also.

User creation through Data Model (https://trello.com/c/wffucRoT)

  • It was fixed two cases related to the flow to create users through Data Model integration. The process now respects the parameter to send the welcome e-mail. It only sends it if it has the setting defined for it.

Carol Connect

2C: Database

MySQL database supporting custom parameters (https://trello.com/c/1qaTWMyq)

  • The MySQL database now supports custom parameters when creating the connection (key/value parameter following MySQL JDBC URL, more details at https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-jdbc-url-format.html).

Reviewed the connection with OpenEdge and Oracle databases (https://trello.com/c/cpep5SEs)

  • Fixed some validations to identify if the Oracle database is a Datasul product. The configuration to the DF file (data structure definition file - related to OpenEdge databases) is not mandatory anymore.

2C: Platform

Carol Connect allows configuring the connection with Carol using the Connector Token (https://trello.com/c/zz3A7c8M)

  • Carol Connect allows setting its connection with Carol using Connector Token (https://tdn.totvs.com/pages/viewpage.action?pageId=552107176). It makes the process more stable since it will not break if the user changes the password. It still supports the user/password configuration.

Carol Connect sending clear CDS command twice (https://trello.com/c/OVmurEKh)

  • Fixed a case to avoid 2C sending twice the command to clear the CDS storage (staging area).

Supporting TOTVS Identity (https://trello.com/c/O0e2rdTw)

  • Carol Connect now supports TOTVS Identity. The process to set it up is available at https://docs.carol.ai/docs/totvs-identity-integration#enabling-carol-connect-with-totvs-identity

Ps.: This release has the sprints 3.18 and 3.19 (including weekly deployments).