About Brightspace Data Sets
Brightspace Data Sets contain raw, user-level data from Brightspace Learning Environment, and can be combined with other Advanced Data Sets and Brightspace Data Sets to provide administrators with the maximum flexibility in aggregating and filtering the data specific to their needs.
To track changes in Brightspace Platform releases, Brightspace Data Sets have their own version number. Specific changes to individual Brightspace Data Sets are captured in their respective topics. For example, if a new field is added to a data set, the topic includes the updated field and version number it was introduced in.
|Brightspace Platform Version||Brightspace Data Sets Version|
6.0 - Contains all changes
5.7 - Contains Grade Scheme Ranges data set; ResultId column in Announcements data set; ActivityType and Version columns in Competency Activities data set; Version column in Competency Activity Results data set; Version column in Grade Schemes data set; Version column in Rubric Assessment Criteria data set; and Version column in Survey Attempts data set.
|20.20.7/July 2020||4.10 / 5.4 - Contains all changes|
|20.20.6/June 2020||4.9 / 5.3 - Contains all changes|
|20.20.5/May 2020||4.8 / 5.2 - Contains all changes|
|20.20.4/April 2020||4.7 / 5.1 - Contains all changes|
5.0 - Contains all changes
4.6 - Contains the ToolId column in Activity Exemptions Log data sets; ToolId, ResultId, DeletedDate, CreatedBy, LastModifiedBy, and DeletedBy columns in Content Objects data sets; and Version column in Discussion Topic User Scores data sets.
Beginning with the 10.7.5/September 2017 release, the Continuous Delivery (CD) Update process automatically rolls out the new Brightspace Data Sets minor version to clients. Minor version updates (such as 4.7 to 4.8) only contain low impact changes that will not break downstream systems or reporting, such as adding a column.
When a new major version (such as 5.X) is released, clients normally have a 6-month opt-in window where they can choose to adopt the new major version; when this window is over, clients are automatically updated. However, when the new version is required to resolve a defect, clients are automatically updated right away.
Once the Brightspace Data Sets version number has been updated, changes appear in the data the next time the Brightspace Data Sets are generated. If the Brightspace Data Sets are in the process of being generated when the version number is changed, any data set created after that change has been made reflects the latest version. So, it is possible that some of the data sets might reflect the previous version, and the rest might reflect the latest version.
For more information about managing Brightspace Data Sets updates, see the Updating The Brightspace Data Sets Change Management Policy on Brightspace Community.
Full Data Sets and Differentials
To analyze and report on the data, Brightspace Data Sets are available in both full data sets and differentials. Full data sets are complete extracts of historical data, refreshed weekly. While full data sets contain a complete view of historical data, they result in large data extracts, taking longer to download. Differential (diff) data sets contain the differences (diffs) of the data that has been updated or inserted in the previous day, and refreshed daily. Daily diffs result in smaller and more up-to-date data extracts, making it faster for administrators to download and process the data.
Note the following about differential data sets:
- To prevent data gaps, updated data sets include some overlap. For example, an updated data set may contain rows from the last few minutes of the previous hour and the next few minutes of the next hour.
- Only inserts or updates display, including updates to existing Is_Deleted columns.
- Only updates on first-class data display. For example, the Quiz Attempts data set displays additional attempts; however, updates to the QuizName field do not display.
Note the following exceptions about Brightspace Data Sets:
- The Content User Progress, Discussion Post Read Status, Discussion Posts, Quiz User Answer Responses, Quiz User Answers, Session History, and User Logins data sets are limited to 3 years of data (all of the previous two calendar years and the current calendar year to date).
- The differentials for Quiz User Answer Responses, Session History, and User Logins are also limited to 3 years of data (all of the previous two calendar years and the current calendar year to date).
- The Pre-Requisite Conditions Met data set is limited to 2 years of data.
- All other Brightspace Data Sets adhere to the default system limit to return 150 million rows of the most recent data.
Scheduling generation of Brightspace Data Sets
For most Brightspace Data Sets, differential data sets generate on a daily basis. In addition to the differential, the full CSV data set continues to generate once a week.
Hourly data freshness add-on
For administrators with high-frequency reporting needs, clients can purchase an add-on for the Data Hub tool that refreshes specific Brightspace Data Sets on an hourly basis.
- For each Brightspace Data Set, differential data sets generate on an hourly basis, and capture any data inserted or updated in the last hour.
- In addition to the differential, the full CSV data set continues to generate once a day.
The add-on is available to AWS-hosted Brightspace Core clients in the following regions:
- North America
To enable the add-on for the Data Hub tool, contact your D2L Account Manager.