REDCap DMZ Updates

Version 12.5.6 (released on 2022-08-19)

CHANGES IN THIS VERSION:

  • New Feature: SendGrid Template Advanced Settings for Alerts & Notifications
  • Introduction- A new “advanced settings” section was added to the Alerts & Notifications interface when building an alert using the relatively new SendGrid Template alert type that gives users more control over the underlying SendGrid API call being made when REDCap triggers a SendGrid Template alert. Note that all of the advanced settings are optional, and they are all disabled by default. If “SendGrid Template email services for Alerts & Notifications” are enabled for a project on the Project Setup page, then these advanced settings will appear in the alert creation dialog after selecting “SendGrid Template” as the alert type. The new advanced settings are all listed in detail below.
  • SendGrid Unsubscribe Groups- SendGrid can allow recipients of its emails to unsubscribe from all emails being sent from a sendgrid account, or from emails associated with specific unsubscribe groups in a sendgrid account. To take advantage of custom unsubscribe groups, you can create unsubscribe groups in your sendgrid account then associate them with alerts in your REDCap project. When a recipient unsubscribes from an email that has been associated with a specific unsubscribe group, they get added to that unsubscribe group's list and any future emails that are associated with that unsubscribe group will not be delivered to them. An alert can be associated with at most one unsubscribe group. Here is SendGrid's documentation on unsubscribe groups: https://docs.sendgrid.com/ui/sending-email/unsubscribe-groups.
  • SendGrid Categories- SendGrid allows you to associate arbitrary categories to each email you send from your account, effectively giving you the ability to tag each individual email sent with different metadata about the email like the email type. Unlike unsubscribe groups, categories don't have to be made in your sendgrid account before associating them with an alert in REDCap. You can define your categories in REDCap as you create your REDCap alert, and your sendgrid account will automatically detect new categories as emails get sent with them. In your SendGrid account's Category Stats page, you'll be able to see data about your emails by category. You can associate up to 10 unique categories per email, and a category name cannot be longer than 255 characters.
  • SendGrid Mail Settings- Full documentation for the SendGrid bypass settings can be found at https://docs.sendgrid.com/ui/sending-email/index-suppressions#bypass-suppressions.
  • Bypass List Management- When enabled, your email will be delivered regardless of any other existing suppression management control in your account. For example, if a recipient is in an unsubscribe group or the global unsubscribe group, they will still receive the email if bypass list management is enabled. Bypass List Management can't be combined with any other bypass option.
  • Bypass Spam Management - Allows you to bypass the spam report list to ensure that the email is delivered to recipients. Some email services allow recipients to mark emails as spam. In some cases, sendgrid will be notified when a recipient marks an email as spam and will maintain a spam report list.
  • Bypass Bounce Management - Allows you to bypass the bounce list to ensure that the email is delivered to recipients. A bounce occurs when a receiving mail server rejects an incoming email. This can happen if the recipient address is bad, for example. If sendgrid sees too many bounces happening, it will add that recipient to a bounce list and it will stop trying to send mail to that recipient. Enabling this will bypass that bounce list and force sendgrid to retry delivery.
  • Bypass Global Unsubscribe Management - When enabled, your email will be delivered even if the recipient is on your account's global unsubscribe list.
  • Sandbox Mode - Sandbox mode lets you check for errors in the SendGrid API call used to send an email without the potential of delivering the email. If you're unsure about your sendgrid configuration, you can run a test by enabling sandbox mode for an alert and triggering it. If your project's logs state that the alert was sent successfully and you don't see any errors, then your configuration is good to go. However, since sandbox mode was enabled for that alert, an email was not actually sent. After you're satisfied with your tests, you can disable sandbox mode and start sending real emails with your alert.
  • SendGrid Tracking Settings
  • Click Tracking - SendGrid has the ability to detect when a recipient clicks on links in an email. The count of clicks for a given email can be seen in the email activity section of your sendgrid account.
  • Open Tracking - SendGrid has the ability to detect when a recipient opens an email by embedding a single pixel image in an email. Enabling this setting will make sendgrid include this tracking pixel in your emails. You can view the count of opens for a specific email in the email activity section of your sendgrid account.
  • Subscription Tracking - If subscription tracking is enabled and configured on your sendgrid account, this setting lets you choose whether or not you want to include the global unsubscribe link associated with the subscription tracking feature in your emails. Note that you can utilize unsubscribe groups without using the more general subscription tracking feature. I believe subscription tracking is disabled by default on a sendgrid account. Here is some documentation from sendgrid about unsubscribe methods: https://support.sendgrid.com/hc/en-us/articles/1260806604209-Unsubscribe-Methods
  • Miscellaneous Additions
  • Added an External Service Check for https://api.sendgrid.com/v3 in the Control Center's Configuration Check page.
  • Added a line in the Modules utilized section of the Systems Statistics page to keep track of how many non-practice projects are utilizing sendgrid for Alerts & Notifications.
  • Additional SendGrid API Token Requirements - To fully support SendGrid Advanced Settings, the SendGrid API token used in the project's setup needs the permission for getting an account's unsubscribe groups through the API. This permission is mapped to the asm.groups.read scope. You can add this permission to your existing API token by editing its permissions in your SendGrid account and giving it Read Access to Unsubscribe Groups in the Suppression section.
  • Improvement:When utilizing Multi-Language Management in a project, the Field Finder on the Codebook page now supports searching in translated field labels.
  • Improvement:The date of the most recent REDCap upgrade for the system is now displayed near the bottom of the main Control Center page. (Ticket #69036)
  • Improvement:"Project 5 (COVID-19)" was added as a new classification that is selectable under the NIH CDE Repository catalog for the Field Bank feature in the Online Designer. Project 5 (COVID-19) is a classification of NIH-Endorsed CDEs (Common Data Elements).
  • Major bug fix:When exporting a PDF that contains a multiple choice field that has been flagged as an Identifier field, if the user has De-Identified data export rights for the field's instrument, the data for the field would mistakenly not be removed from the resulting PDF. (Ticket #132190)
  • Major bug fix:When clicking the “Forgot your password?” link on the login page and then entering the username of a valid REDCap user, the password of the username entered would mistakenly be reset immediately after being entered, which could lock out the user if a malicious user is randomly entering usernames to try and discover a valid username. It now only resets the user’s password after they click the password reset link in the email that they receive. Additionally, in order to prevent malicious users from discovering valid usernames, the password reset page now returns the exact same message in all situations, whether the username entered is a real username or not. In the case when using one of the “X & Table-based” authentication methods, if the user entered is an external user (i.e., not a Table-based user), they will also receive an email that will inform them that they must reset their password using an external resource outside of REDCap (or it will instead display the custom password reset text that has been defined in the Control Center). (Ticket #132595)
  • Major bug fix:When using certain external authentication methods, survey pages might sometimes mistakenly time out if the project's internal Record List Cache (a secondary list of records in the database for improving performance) had not been built yet, which is done automatically by REDCap internally. This would cause an internal API call to fail when it is made inline while loading survey pages, thus causing the survey page not to load. (Ticket #104761)
  • Bug fix: The Codebook page can become very slow in certain situations when lots of fields exist in the project, especially when utilizing languages for Multi-Language Management. (Ticket #132349)
  • Bug fix: When using Multi-Language Management in a project, some translations might get mistakenly overwritten when importing a CSV/JSON translation file due to an issue with case sensitivity with the language ID (e.g., “es” vs “ES”). (Ticket #132443)
  • Bug fix: Some of the text inside the dialog displayed to an administrator when a project has been marked as Completed was changed in order to be less confusing about the project's status after the admin has restored it. (Ticket #132499)
  • Bug fix: When using Azure AD authentication, users might mistakenly not have their first/last name and email auto-populated into their user profile after initially logging in to REDCap. This bug was supposedly fixed in the previous version but mistakenly was not. (Ticket #130664b)
  • Bug fix: When using the Data Resolution Workflow feature and creating data queries based on the results of Data Quality rules, the results of the Data Quality rules might not display the correct number of comments for a given discrepancy unless it belongs to a repeating instrument. (Ticket #131878)
  • Bug fix: When a user's date/time format user preference on the Profile page is set specifically to "YYYY-MM-DD and 24-hour time", some timestamps displayed in the REDCap user interface (e.g., Most recent activity on Project Home, Email Logging sent time) would mistakenly display the "seconds" component of the datetime when it should only display hours and minutes. (Ticket #132678)
  • Bug fix: When using Azure AD authentication, the username for a B2B collaboration user object might contain an "#EXT#" identifier as text inside it in certain cases. This is problematic to have the character "#" in a user's username. If this occurs, the text "#EXT#" will be automatically removed from the user's username. (Ticket #121605b)
  • Bug fix: By manipulating URLs and/or JavaScript variables on a REDCap project page, a user might be able to request an API token for a project in which they do not explicitly have API rights (although they would have to have access to the other project in order to do this). Even if the administrator approved the token request via the To-Do List or via the email request, the user would not be able to obtain the API token that was created for them, nor would they be able to use the token even if they could somehow obtain it. So no real harm or privacy issues could result from this. (Ticket #132778)
  • Bug fix: When using Multi-Language Management and importing translations for survey settings via a CSV file, some survey settings would mistakenly fail to import successfully. (Ticket #132828)
  • Bug fix: When using Multi-Language Management, the “[Reminder]” text for Automated Survey Invitation reminders was mistakenly not translatable. It can now be translated on the User Interface > Survey > Survey Emails section on the MLM setup page. (Ticket #132868)

Version 12.1.2 (released on 2022-01-14)

CHANGES IN THIS VERSION:

  • Bug fix: A project dashboard with custom access settings might mistakenly not be accessible to administrators using the "View project as user" feature.
  • Bug fix: When creating or editing a Project Dashboard that has been set as "public", the option to create a custom public link would mistakenly not be displayed on the page (assuming that the URL Shortening Service is enabled at the system level).
  • Bug fix: When creating or editing a report that has been set as "public", the option to create a custom public link would mistakenly not be displayed on the page (assuming that the URL Shortening Service is enabled at the system level).
  • Bug fix: If using the @CALCTEXT action tag on a datetime field, in which the [survey-time-completed] Smart Variable is referenced inside @CALCTEXT(), the resulting value might cause the calculation error popup to display on a survey or data entry form, and the value might not save correctly on the form/survey, via data import, or via running Data Quality rule H. This issue occurs mostly when using field validation with formatting H:M (rather than H:M:S) and also with formatting MDY or DMY (rather than YMD).
  • Bug fix: If a date or datetime field was using the @HIDEBUTTON action tag, the date format label (e.g., "M-D-Y") would mistakenly not be displayed on the right.
  • Bug fix: The Concurrent Users chart on the Activity Graphs page in the Control Center would mistakenly not display all past data in the chart. (Ticket #120921)
  • Bug fix: PHP error occurs for PHP 8.0 or 8.1 when downloading Automated Survey Invitations as a CSV file in the Online Designer. (Ticket #120965)
  • Bug fix: When piping data on an instrument for a field from another instrument while also using the Multi-language Management feature for the current instrument, the piped value might mistakenly not display on the page.
  • Bug fix: When performing calculations for a @CALCTEXT field (whether on a data entry form, survey page, data import, or Data Quality rule H), some dynamically-created regular expressions in PHP that search for other calculated fields or @CALCTEXT fields that are used within the original @CALCTEXT field might cause an overload due to the regular expression being too long, thus possibly resulting in not accurately determining dependent fields used inside the @CALCTEXT field's equation. This means that some @CALCTEXT fields might possibly not get their value updated successfully.
  • Bug fix: Some long-running reports might mistakenly return the error message "An unknown error has caused the REDCap page to halt..." in specific edge cases.
  • Bug fix: If an alert has an [aggregate-X] Smart Variable piped into the alert's email body, it might cause the cron job to crash when attempting to send the alert. (Ticket #120561)

Version 12.1.1 (released on 2022-01-10)

CHANGES IN THIS VERSION:

  • Major bug fix: The new "Time (HH:MM:SS)" field validation might not have been stored correctly (and thus would not work successfully) if you previously upgraded to REDCap 12.1.0.
  • Major bug fix: Some installations (depending on MySQL/MariaDB version) might mistakenly have a database structure issue involving the table "redcap_log_view_requests" after upgrading to REDCap 12.1.0. (Ticket #120622)
  • Bug fix: The field drop-down for the "Designate a Secondary Unique Field" setting in the "Additional Customizations" popup on the Project Setup page would mistakenly not include some Textbox fields (notably those with no Action Tags or Field Annotation).
  • Bug fix: When using Smart Variables that utilize the parameters ":fields" or ":instrument" in a calculated field or @CALCTEXT field, if the user is entering data on a form or survey, the calculation might mistakenly not get updated if fields used inside the Smart Variable exist on a different instrument or event.
  • Bug fix: For certain server configurations, the REDCap cron job might mistakenly crash due to a floating point precision issue when creating a timestamp. This occurrence is fairly rare. (Ticket #120688)
  • Bug fix: When using certain Smart Variables inside a calculation or @CALCTEXT field, a calculation error message might mistakenly appear on the data entry form or survey page and thus would prevent calculations from occurring on that page. (Ticket #120660)
  • Bug fix: When a report contains data from a repeating instrument and/or repeating event, in which the report's checkbox setting "Include the repeating instance fields (redcap_repeat_instrument, redcap_repeat_instance) in the report and data export?" is not checked, viewing the Stats & Charts page for the report would display the charts and tables correctly unless a user selects a Live Filter for the report, in which it would mistakenly cause all/most tables and charts not to display at all on the page. (Ticket #120408)

Version 12.1.0 (released on 2022-01-07)

CHANGES IN THIS VERSION:

  • New feature: Conditional logic for Survey Auto-Continue - When enabling Survey Auto-Continue on the Survey Settings page for a survey, users may now optionally specify conditional logic to determine whether or not the auto-continue should be applied. As such, REDCap will auto-continue to the next survey *only* if the conditional logic is TRUE or if the logic textbox has been left blank. This new option can be used as a simpler alternative to the Survey Queue, which can require more complex instrument-event level configurations for longitudinal projects.
  • New feature: Dynamic min/max range limits for fields - Instead of using exact values as the minimum or maximum range of Textbox fields (e.g., "2021-12-07"), you may now also use "today" and "now" as the min or max so that the current date or time is always used. These can be used to prevent a date/time field from having a value in the past or in the future. Additionally, you can now pipe a value from another field into the field's min or max range setting - e.g., [visit_date] or [event_1_arm_1][age]. This can help ensure that a Textbox field (whether a date, time, or number) has a larger or smaller value than another field, regardless of whether the field is on the same instrument or not.
  • New action tag: @FORCE-MINMAX - The action tag @FORCE-MINMAX can be used on Textbox fields that have a min or max validation range defined so that no one will not be able to enter a value into the field unless it is within the field's specified validation range. This is different from the default behavior in which out-of-range values are permissible. Note: @FORCE-MINMAX is also enforced for data imports to ensure the value is always within the specified range.
  • New field validation: "Time (HH:MM:SS)" - This new time-based field validation (unique name "time_hh_mm_ss") will be added automatically and enabled by default during the upgrade process. This validation forces users/participants to enter a time value that contains the hour, minute, and second components. It also includes the usage of the "Now" button and the timepicker popup widget, both of which are displayed next to the field on the survey page or data entry form. Note: Fields with this field validation can be utilized inside the datediff() function. (Thanks to the Field Validation Committee for this addition.)
  • Medium security fix: A Cross-site Scripting (XSS) vulnerability was discovered where a malicious user could potentially exploit it by inserting HTML tags and/or JavaScript event attributes in a very specific way as user-defined text in various places.
  • Minor security fix: If a field contains integer values (e.g., Textbox, Radio, Drop-down) for a record, and then the field is changed to be a File Upload field, viewing a data entry form or a report that contains that field might (depending on the pre-existing integer value of the field) mistakenly expose the filename of files that have been uploaded to other File Upload fields, including possibly those from other projects. Users are not able to download these uploaded files or view their contents, but can view the filename of the file on a data entry form or a report.
  • Minor security fix: A Blind SQL Injection vulnerability was found on the Cron Jobs page in the Control Center, in which a malicious user could potentially exploit it by manipulating an HTTP request on that page.
  • Minor security fix: A Cross-site Scripting (XSS) vulnerability was discovered where a malicious user could potentially exploit it by inserting HTML tags and/or JavaScript event attributes in a very specific way into the URL on the API Tokens page in the Control Center and also on the API page in a project.
  • Major bug fix: In a longitudinal project with Data Access Groups, importing data via the "Import Records" API method for an existing record that is assigned to a DAG, in which the API parameters format="json" and overwriteBehavior="overwrite" are used, if the JSON data being imported contains a non-blank value for the "redcap_data_access_group" field for one event while another event of data (for the same record) does not contain the "redcap_data_access_group" field at all in the JSON, REDCap would mistakenly perceive the absent "redcap_data_access_group" field as a blank value and thus would un-assign the record from the DAG (due to the overwriteBehavior="overwrite" parameter being used). When this occurs, the DAG unassignment event would also not get logged on the project Logging page.
  • Improvement: For projects using the Clinical Data Interoperability Services (CDIS), a new observation category “social history” was added for both CDM and CDP projects, thus allowing them to import this new type of EHR data into REDCap.
  • Improvement: New CDIS panel on the left-hand project menu to display information and links that are relevant to projects using either Clinical Data Pull or Clinical Data Mart.
  • Change: When using the Auto-adjudication feature in a Clinical Data Pull (CDP) project, in which it has been set to notify the user via REDCap Messenger whenever a record has been auto-adjudicated by the system, REDCap now automatically deletes all previous auto-adjudication Messenger threads for this project for the user. In previous versions, the user might receive thousands of Messenger notifications, which could cause REDCap itself to become sluggish for the user. Now it only keeps the latest notification for the user.
  • Various updates and changes to the External Module Framework, including a slight change to the EM link on the left-hand project menu (i.e., the "External Modules" link was replaced with "Manage" further down the project menu).
  • Change: In the database backend, the “redcap_log_view” database table will be renamed to “redcap_log_view_old”, and an empty replacement table (named “redcap_log_view”) will be created in its place. The old table and its contents will no longer be used in the application except for very specific, seldom-used functionality (e.g., viewing Page View events on a project’s Logging page). The new table will have a slightly different structure, such as a BIGINT primary key (instead of INT) and better/more indexes to improve query performance for the table. The retiring/renaming of the old table should not have any effect on plugin/hook/module developers unless you are performing direct queries on the “redcap_log_view” table to pull information from months or years in the past, in which case you would want to also query the “redcap_log_view_old” for such information. Note: During the upgrade process, the last 30 minutes worth of activity from redcap_log_view will be automatically transferred to the new table in order to maintain continuity within the application for before and after the upgrade, especially if the system is not taken offline during the upgrade.
  • Bug fix: Drop-down fields using the auto-complete option would cause the webpage to be slow/laggy when typing a value into the field's textbox or when clicking the down-arrow button for the field to view the full list of choices if the field has hundreds or thousands of choices defined. This slowness was due to the auto-complete feature not being set up correctly in the underlying JavaScript. Note: Clicking the down-arrow button for an auto-complete drop-down with 1000+ choices when the field has no value will now display a notice next to the field that the full list of choices cannot be displayed and instead encourages the user to type a value to search all options.
  • Bug fix: When referencing a Smart Variable inside conditional logic (e.g., Data Quality rules, ASI logic) in which the Smart Variable is appended with a colon+parameter while also being prepended with a unique event name (e.g., [event_1_arm_1][survey-date-completed:form_1]), the logic might fail to be successfully evaluated. This could cause Data Quality rules to throw an error or could cause survey invitations for ASIs not to get sent in specific cases. (Ticket #120543)
  • Bug fix: When a multi-page survey contains required fields that exist on pages after page 1, in some specific scenarios it might mistakenly display the "Some fields are required!" prompt for fields on later pages after submitting the first page. Note: The participant would still be allowed to continue to the next page after the initial submission of page 1. (Ticket #120518)

Version 12.0.7 (released on 2021-12-28)

CHANGES IN THIS VERSION:

  • Security improvement: Any third-party (i.e., external service) API keys/secrets that are currently stored in the redcap_config database table via a System Configuration page in the Control Center (e.g., AWS S3 secret key, Twilio Auth Token for two-factor authentication) will now have its value stored in encrypted format in the redcap_config table instead of being stored as plain text. This will occur automatically and transparently after upgrading. This will prevent anyone from obtaining these keys/secrets if they view the contents of the redcap_config table.
  • Minor security fix: Updated “Axios” third party JavaScript package due to reported vulnerabilities.
  • Change: The dialog that is displayed when editing a field's branching logic in the Online Designer, in which one or more fields have the exact same branching logic as the current field, contains different text to better explain what clicking "Yes" will do.
  • Bug fix: When using specific configurations of the Survey Queue while running a specific PHP version on the REDCap web server (PHP 8.0 or 8.1?), it might cause the survey page to suddenly crash with a fatal PHP error after completing a survey. (Ticket #120211)
  • Bug fix: A calculation error would occur (displaying the error popup) on a survey page or data entry form if the @CALCDATE action tag is used on an MDY or DMY formatted date or datetime field, in which the first parameter of @CALCDATE contains an if() function where the first field used inside the if() is not a date or datetime field. (Ticket #119510)
  • Bug fix: When an Ad Hoc calendar event is viewed in the calendar popup in a longitudinal project, it would mistakenly display the instruments designated for the first event in the Data Entry Forms list inside the calendar popup. Ad Hoc events should not display any forms in the calendar popup. (Ticket #120224)
  • Updates and various fixes for the External Module Framework, such as the following: Fixed multiple issues with survey & NOAUTH CSRF protection, Added support for hidden subsettings, Improved log display performance, and Added project IDs to error emails.
  • Bug fix: [scatter-plot] Smart Charts might not display their x-axis in correct numeric order for slider fields or some other fields with numeric data. Additionally, for this same situation [line-chart] Smart Charts might mistakenly display their x-axis as a categorical-type display rather than a linear-type display. (Ticket #120214)

Version 12.0.6 (released on 2021-12-23)

CHANGES IN THIS VERSION:

  • Change/improvement: New CDIS setting - “Identity provider (optional)” - If specified on the Clinical Data Interoperability Services page in the Control Center, the identity provider will be used in the OAuth2 authorization process to identify the server that will exchange the FHIR access token with REDCap. This setting should only be set if the real FHIR base URL of the EHR system is different from the one specified on this page (e.g., the EHR system is behind a proxy).
  • Bug fix: If database table structure issues exist, in which REDCap provides the SQL to fix the issue, the generated SQL might fail when executed on some versions of MySQL/MariaDB if the SQL contains queries to drop Primary Keys that are being used as Foreign Keys in other tables. The generated SQL now includes queries to drop the Foreign Key before dropping the Primary Key, and then also the SQL to re-add the Foreign Key after fixing the Primary Key.
  • Bug fix: When using Clinical Data Pull or Clinical Data Mart and utilizing the “Break the Glass” feature, an authentication error might occur when attempting to use one’s credentials to break the glass of a patient record, specifically when using LDAP authentication.
  • Bug fix: When using the ":value" modifier when piping a field value while also referencing the unique event name and an X-instance Smart Variable (e.g., [c_hmcadrc_visit_re_arm_1][cog_behav_status:value][last-instance]), the label of the multiple choice field option mistakenly might get piped instead of the value of the selected choice. (Ticket #119879)
  • Bug fix: Depending on the naming conventions of the records in the project, the records in the record drop-down list on the "Add/Edit Records" page might appear slightly out of order if Record Auto-Numbering was enabled after non-numerical record names had already been created in the project.
  • Bug fix: The @RICHTEXT action tag would mistakenly not work on survey pages. (Ticket #119996)
  • Bug fix: When making a call to REDCap::saveData() or to the "Import Records" API method to import record data for records that have been assigned to a Data Access Group, if the data being imported is for a longitudinal event that currently has no data for the record, then the project's Logging page might mistakenly denote the record as being created during the import process, despite the fact that the record already exists and has data in other events. In some very rare cases, this might additionally cause the record to get unassigned from its current DAG with no logging to indicate that this happened.
  • Bug fix: Fields with the @CALCDATE or @CALCTEXT action tags could mistakenly be chosen as the Secondary Unique Field in the project, although this should not be allowed because it could cause the field not to perform its calculation correctly, especially if the field exists on a repeating instrument/event. As calc fields have never been allowed for use as the Secondary Unique Field, neither should @CALCDATE or @CALCTEXT fields. (Ticket #119773)
  • Bug fix: Fields with the @CALCTEXT action tag might mistakenly (in specific situations) return an incorrect result if values with leading zeros are utilized in the equation, in which the value "007" would be returned as "7". This would mostly occur when evaluating radio or drop-down fields that have leading zeros for one or more choice codes but do not have any choice codes that contain letters. (Ticket #120024)

Version 12.0.5 (released on 2021-12-17)

CHANGES IN THIS VERSION:

  • New feature: New design for the “Help & FAQ” page.
  • New Smart Variable: [event-number] - The current event's ordinal number as listed on the Define My Events page that denotes the order of the event within a given arm. (Ticket #70973)
  • Improvement/change: The Define My Events page now displays a new column to display each event's Event ID number. Also, the Smart Variable corresponding to each column in the table on the Define My Events page (e.g., [event-number], [event-label) are displayed in small gray text below the header text in the table to help users more easily learn where the values of those Smart Variables originate. (Ticket #115791)
  • Improvement: When using OAuth2 Azure AD Authentication, you may now specify a different AD attribute whose value determines the REDCap user's username. By default, it uses the AD attribute "userPrincipalName", which often resolves to the user's email address. The Security & Authentication page has a new drop-down setting to allow admins to alternatively specify the AD attribute "samAccountName", which would resolve to something like "pharris", for example. This provides an option if the institution prefers not to use a user's email address as their REDCap username. Note that this setting does not change the Azure AD login name, which is still the user's email address / userPrincipalName. Administrators may want to select the samAccountName to help retain account usernames when transitioning from LDAP to Azure AD, or if samAccountName is considered an immutable (and thus more reliable) user ID at your institution.
  • Change: Although REDCap sets the cookie "samesite" attribute to "Lax" by default, the "samesite" attribute can be overridden by adding the following line of code in the REDCap database.php file on the web server: $GLOBALS['cookie_samesite'] = "None"; // Possible values: "None", "Lax", or "Strict".
  • Bug fix: After a participant clicks the "Save & Return Later" button on a survey and then attempts to send themselves the survey link for returning, the resulting confirmation dialog titled "Email sent!" would mistakenly have the word "undefined" inside the dialog rather than the correct stock language text "The email was successfully sent to:". (Ticket #119438)
  • Bug fix: Various JavaScript-driven messages displayed on data entry forms and survey pages would mistakenly display "undefined" instead of the correct text.
  • Bug fix: REDCap now automatically sets mysqli_report to OFF for better compatibility with PHP 8.1, which defaults this setting to MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT. Without setting this, PHP might fail with a fatal error whenever a query to MySQL fails, but this only occurs for certain configurations of PHP 8.1.
  • Bug fix: Typo in Shibboleth authentication settings in the Control Center.
  • Bug fix: When using OAuth2 Azure AD Authentication, the usernames set on the Security & Authentication page for Primary Admin and Secondary Admin were case-sensitive. They are now case-insensitive so that the admin usernames can be entered in any case and will still work.
  • Bug fix: When using certain versions of MySQL or MariaDB, the Easy Upgrade or Automatic Upgrade features might mistakenly not be allowed, in which REDCap might think that the REDCap MySQL user does not have "DROP" privileges for the database. (Ticket #119577)
  • Bug fix: If records are named a specific way in a project, they might appear out of order when displayed in certain contexts, such as if the record list spans multiple pages on the Record Status Dashboard. (Ticket #119189b)
  • Bug fix: When uploading an allocation file on the Randomization page, it might mistakenly allow the user to upload multiple allocation files while on the same page. This should not be allowed. (Ticket #119640)
  • Bug fix: When using the Multi-language Management feature to translate the choice labels of Yes/No and True/False fields, those choice labels would mistakenly not appear translated in downloaded PDFs of an instrument (both with and without data).
  • Bug fix: If a '<>' operator is used in a field's Field Annotation/Action Tags, then the operator would mistakenly not be displayed in the Codebook. (Ticket #119705)
  • Bug fix: In situations where slider fields should be disabled on a data entry form (e.g., user has read-only Data Viewing Rights for the instrument), sliders could mistakenly become editable on the page if clicked. Note: Since the user cannot submit the page in this situation, it does not affect data, but can be confusing. (Ticket #119760)
  • Bug fix: When utilizing [aggregate-X] Smart Variables in a calculated field or @CALCTEXT field, if the user is entering data on a form or survey, the calculation might mistakenly not get updated if fields used inside the [aggregate-X] Smart Variable exist on a different instrument or event. However, the calc/@CALCTEXT field would get correctly updated when running Data Quality rule H or when performing a data import.
  • Bug fix: An error message would mistakenly be displayed when attempting to pipe a field variable into the "Redirect to a URL" textbox on the Survey Settings page.
  • Change: Added user's REDCap username to the email subject for REDCap Messenger email notifications to help distinguish them if the same primary email address is used for multiple users.
  • Bug fix: The cron job that sends email notifications for REDCap Messenger might mistakenly send multiple emails repeatedly to users. (This is an additional fix to the same bug fix from one month ago.)
  • Bug fix: When using the Data Resolution Workflow, the DRW dialog would mistakenly not allow the user to reassign the data query to another user if the data query had been opened immediately after the field's data value had been "Verified". (Ticket #119758)

Version 12.0.4 (released on 2021-12-10)

CHANGES IN THIS VERSION:

  • Change/improvement: A link to the "Language File Creator/Updater" page was added to the Control Center's left-hand menu in the Administrator Resources section.
  • Change/improvement: When printing a report, the "Number of results returned" and "Total number of records queried" counts are now included in the printout of the page.
  • Bug fix: The "RTL" dialog on the Multi-Language Management page in the Control Center would mistakenly be empty instead of having the appropriate text.
  • Bug fix: A PHP fatal error would be thrown when attempting to edit a field in the Online Designer if using PHP 7.2. (Ticket #118919)
  • Bug fix: When using newer versions of MySQL or MariaDB, the Easy Upgrade or Automatic Upgrade features might mistakenly fail in certain instances if the REDCap MySQL user does not have "REFERENCES" privileges for the MySQL database. (Ticket #119033)
  • Bug fix: When pulling EHR data from the Conditions R4 endpoint for Clinical Data Pull or Clinical Data Mart, the condition’s date value might mistakenly fail to get imported into the REDCap project.
  • Changes and various bug fixes for the External Module Framework, including the following: Included cron start & end times in the cron log, Improved unit testing & psalm scanning (of the framework itself), and Improved performance of the "Logs" page.
  • Bug fix: The variable name displayed for fields on the Codebook page would mistakenly display a square bracket after the branching logic instead of before it. (Ticket #119302)
  • Bug fix: If a user is in a Data Access Group, the Participant List would display an incorrect count of how many visible participants are in the Participant List, and it might show some pages of the Participant List as being empty. (Ticket #119056)
  • Bug fix: If records are named a specific way in a project (e.g., ABC-1, ABC-2), they might appear out of order when displayed in certain contexts, such as if the record list spans multiple pages on the Record Status Dashboard. (Ticket #119189)
  • Bug fix: If a calculated field is using a datediff() function with a datetime field and with "today" as the first two parameters, it would mistakenly throw an error on the page that a calculation error exists. (Ticket #119049)
  • Bug fix: When sending an SMS via Twilio, in which the Twilio API returns the error message "violates a blacklist rule", the survey invitation log would mistakenly not flag this error correctly with reason_not_sent = 'PARTICIPANT OPTED OUT' but instead would revert to the default reason_not_sent of 'ERROR SENDING SMS'.
  • Bug fix: If HTML tags are used inside the Custom Labels for Repeating Instruments, whenever the dialog is reopened to edit the Custom Labels for Repeating Instruments, the HTML tags will have been automatically removed. It should not remove the HTML tags that have been already saved. (Ticket #119244)
  • Bug fix: When clicking the "export" link to download the results after running Data Quality rule A or B, it would be impossible to determine which field had the missing value for a given row/record if more than one field had a missing value for the whole set of results exported. To remedy this issue, the export file no longer lists each variable name as a separate column (like other DQ rules) but instead has a new "field" column that will list the variable name of the field with the missing value in each row. (Ticket #119276)
  • Bug fix: When upgrading to REDCap 12.0.0 or higher and when the Form Render Skip Logic external module is being utilized for one or more projects, the upgrade script to auto-migrate all the FRSL settings into the new Form Display Logic feature might be slightly incorrect for some FRSL configurations (only affecting longitudinal projects). If the FRSL checkbox setting "Restrict this rule to specific events" is not checked but one or more events have been selected (which is not expected), the resulting behavior from the Form Display Logic would cause the form to be disabled for the selected event, whereas the FRSL module beforehand would disable the form on every event. The auto-migration script now has been changed to match the behavior of the FRSL module for this particular misconfiguration of the FRSL module. (Ticket #118353)
  • Change: For new REDCap installations, the global setting "Minimum number of data points required to display Smart Charts, Smart Tables, or Smart Functions" has been changed from "11" to "5" since the previous default value was regarded as too conservative by many. For existing installations, this value can easily be changed on the User Settings page in the Control Center and additionally can be overridden for any project via the Edit A Project's Settings page.
  • Bug fix: When displaying Smart Charts on a public Project Dashboard, in which the chart is grouped via a secondary field, in some specific cases where data is missing for the first field in the chart but not for the grouping field, the chart might mistakenly get displayed (instead of displaying the message "[INSUFFICIENT AMOUNT OF DATA FOR DISPLAY]") even when it does not meet the minimum data point criteria. (Ticket #119348)
  • Bug fix: Custom Data Quality rules whose logic utilizes fields from repeating instruments might mistakenly return results that are duplicates or not relevant, such as displaying the base/non-repeating instance when all the fields in the logic exist on a repeating instrument. (Ticket #72996)

Version 12.0.3 (released on 2021-12-03)

CHANGES IN THIS VERSION:

  • Major bug fix: When the "Enable support for Survey Auto-Continue" option is checked in the Form Display Logic setup dialog, the feature might mistakenly fail to evaluate the logic correctly during the Survey Auto-Continue process. Thus, it could cause some surveys to get skipped unintentionally.
  • Improvement/change: When using Multi-Language Management on a survey, the current language name is now displayed next to the globe icon at the top right of the survey page so that participants more intuitively understand what the current language is and to click it to change the language.
  • Improvement/change: The Online Designer now denotes whether a field on the instrument contains embedded fields inside its label, choices, notes, etc. by displaying a blue box saying "Contains embedded fields", similar to the green "Field is embedded elsewhere on page" boxes for embedded fields themselves. This will provide users with visual cues to know when and where field embedding is occurring.
  • Improvement: The Design Checker feature for Clinical Data Mart now has improved descriptions of changes that will be made, including the severity of the design issue.
  • Bug fix: When using vertical sliders on forms/surveys, the “Change the slider above to set a response” text would have a translucent background that might mistakenly cover part of the text field displaying the number value. (Ticket #118330)
  • Bug fix: When using the Sponsor Dashboard or Browse Users->View User List By Criteria pages and clicking the "Time of latest password reset" link on the page, the resulting error message might be confusing if the user selects users in the table in which none of those select users log in via Table-based authentication (assuming the system authentication is LDAP+Table or Shibboleth+Table). More text has been added to the error message to inform the user that at least one Table-based authentication user must be selected in order to perform this action. (Ticket #118200)
  • Bug fix: If an admin has "Modify System Configuration Pages" admin rights but does not have "Access to all projects and data with maximum user privileges" admin rights, then if the system was taken offline, the admin would mistakenly not be able to restore the system back to online status. (Ticket #118540)
  • Bug fix: The “Save your changes?” prompt that is displayed when attempting to leave a Data Entry Form via closing the current window/tab might mistakenly cause a JavaScript error rather than displaying the prompt.
  • Bug fix: When using Missing Data Codes for an embedded field with the ":icons" parameter set (e.g., {field1:icons}), the list of Missing Data Codes would fail to display after clicking the "M" icon for the embedded field. (Ticket #118636)
  • Bug fix: When using Missing Data Codes for an embedded field with the ":icons" parameter set (e.g., {field1:icons}), in which the field is a Radio Button field, if the user clicks the "reset" link to reset the value of the field, it would mistakenly throw a JavaScript error. It would still correctly remove the value of the field and reset it, but it would appear to the user as if it did not.
  • Bug fix: The Smart Variables [survey-time-completed] and [survey-date-completed] might not get evaluated correct when used in Survey Queue conditional logic. (Ticket #118452)
  • Bug fix: When attempting to save a custom Record Status Dashboard in a non-longitudinal project, in which one or more instruments are selected for the "Select instruments" option, it would fail to save the selected instruments, thus resulting in displaying all instruments on the custom dashboard instead of only the selected ones.
  • Change: To the right of the REDCap/PHP/MySQL versions listed at the top of the main Control Center page, a "copy" icon was added to allow administrators to easily copy those that version information text so that they may paste them elsewhere, such as when posting a question or bug report on REDCap Community.
  • Bug fix: When a multi-arm longitudinal project does not have "arm 1" defined but has higher-numbered arms defined, it can cause certain things not to work correctly, such as branching logic, calculations, or action tags.
  • Change: In the "Add Field"/"Edit Field" dialog in the Online Designer, it is no longer possible to tab into the Action Tags text box. This was changed because users found it a bit jarring for the Logic Editor dialog to automatically display as they are tabbing through the fields inside the "Add Field"/"Edit Field" dialog.
  • Change: Light gray square brackets are now displayed around the variable name for each field on the Data Dictionary Codebook to aid users when searching for a specific field on the page (because it may sometimes be hard to find a field on the page if it is used in lots of branching logic or calculations).
  • Bug fix: When attempting to do a fresh install of REDCap on PHP 8.0, the install page might mistakenly crash with a blank white page.
  • Bug fix: When a public survey is completed and the "Save & Return Later" feature is not enabled for the survey, references to the survey link via the Smart Variable [survey-link] might mistakenly allow participants to return to the completed survey when instead it should prevent them and thus display the "Thank you for your interest, but you have already completed this survey" message. This could cause further confusion if a participant attempted to download a file for a File Upload field on that survey, in which it would prevent them from downloading it (via an error message); however, this might be confusing since the participant could access the survey page (via this bug) but not the downloadable file on the survey. (Ticket #118314)

Version 12.0.3 (released on 2021-12-03)

CHANGES IN THIS VERSION:

  • Change: The Control Center now recommends using PHP 7.4, 8.0, or 8.1, which are the only currently supported versions of PHP (by the PHP Team).
  • Bug fix: The "Add/Edit Records" page would display a green button with the incorrect text "Add new record for the arm selected above" for projects that do not have multiple arms. The button instead should say "Add new record".
  • Bug fix: When upgrading from a version prior to REDCap 11.4.1, the upgrade SQL script might mistakenly fail when dropping an index on the `redcap_user_roles` table.
  • Bug fix: When copying a project where Twilio is enabled, the various Twilio configuration settings would mistakenly not get copied. Note: The Twilio feature will still be disabled in the newly created project. (Ticket #118265)
  • Bug fix: When using a Project Bookmark as an "Advanced Link", the API call that should return the various parameters (e.g., username, project_id) would mistakenly default to "xml" as the return format when instead it should default to "csv" if the "format" API parameter is not provided in the API request.

Version 12.0.1 (released on 2021-11-23)

CHANGES IN THIS VERSION:

  • Major bug fix: When using Twilio SMS or Voice Call functionality on a survey, field labels or section headers might mistakenly not get included in the SMS message or Voice Call message unless one or more languages have been defined and are active on the Multi-Language Management page.
  • Bug fix: When using Twilio SMS or Voice Call functionality on a survey, the choices for some multiple choice fields would mistakenly not appear in the correct translated language when one or more languages have been defined and are active on the Multi-Language Management page.
  • Bug fix: When using Twilio SMS or Voice Call functionality on a survey, the survey instructions and completion text might mistakenly not appear in the correct translated language when one or more languages have been defined and are active on the Multi-Language Management page.
  • Bug fix: Some rare adaptive PROMIS instruments that contain checkbox or textbox field types (e.g., PROMIS Sexual Function v2 Brief Profile (Female)) would crash in certain instances and prevent the participant from completing the survey whenever a participant attempts to answer a checkbox or textbox field on the survey page.

Version 12.0.0 (released on 2021-11-22)

CHANGES IN THIS VERSION:

  • New feature: Multi-Language Management
    • Summary: Users can create and configure multiple display languages for their projects for surveys, data entry forms, alerts, survey invitations, etc. Users can design data collection instruments and have them be displayed in any language that they have defined and translated so that their survey participants or data entry persons can view the text in their preferred language. This eliminates the need to create multiple instruments or projects to handle multiple languages. NOTE: The MLM feature will not auto-translate text, but provides tools so that users may easily translate them themselves.
    • Usage: When entering data on a data entry form or survey, users and participants will be able to choose their language from a drop-down list or buttons on the page to easily switch to their preferred language for the text displayed on the page. This feature allows users to translate all text related to the data entry process, both for surveys and for data entry forms. Even various survey settings and email text can be translated. For users on data entry forms, if a language is selected, that selection is stored in the user’s user account settings internally (in the REDCap backend database), whereas a survey participant’s selected language will be stored in a cookie in their web browser as a way to remember their language preference if they return in the future (and also to maintain their selected language from page to page). The language can be pre-selected for a participant, if desired, using the “Language preference field” setting on the MLM page in the project or via the @LANGUAGE-FORCE action tags (seen below).
    • User Rights: Users must have Project Design/Setup privileges in a project in order to see the link to the Multi-Language Management page on the left-hand menu.
    • System-level Configuration: The MLM feature can be completely disabled at the system level, if desired, via the MLM page in the Control Center (on the Settings tab). On this page in the Control Center, admins can optionally seed any User Interface (i.e., stock language) translations for the entire REDCap installation, in which users could import any activated User Interface translations into their project. This will only import the User Interface elements (since those are universal to each project), but it can be a big time saver to prevent the user from having to translate those common elements in their project. These can be imported via the Create New Language process in a project (or via the Edit Language setting also).
    • Note: The MLM feature works seamlessly with SMS messages sent via Twilio. Additionally, the MLM feature works with the e-Consent Framework, in which the archived PDF of the participant’s consent form will be stored in the File Repository in the same language in which the participant took the survey.
    • Note: When a project is in production, the MLM page and all translations can only be modified when the project is in Draft Mode. So if the user desires to make edits or additions to their translations, they must first enable Draft Mode via the Online Designer, and then return to the MLM page to make translation changes while in Draft Mode. When the drafted changes are approved, their translation changes made while in Draft Mode will automatically be approved together with them.
      • New Action Tags for Multi-Language Management
        1. @LANGUAGE-CURRENT-FORM - Allows you to capture the currently used language in projects where multilingual data is enabled on data entry forms. The @LANGUAGE-CURRENT-FORM action tag can be used on fields of type 'Text Box' (no validation), and 'Drop-down List', or 'Radio Buttons' (these need to have choices whose codes correspond to the IDs of the defined languages - e.g., 'en'). This action tag is only active on data entry forms and will always, when possible, set the field's value to the currently active language.
        2. @LANGUAGE-CURRENT-SURVEY - Same as @LANUGAGE-CURRENT-FORM, but works only on survey pages. For multi-page surveys, @LANGUAGE-CURRENT-SURVEY needs to be used on a field of each page where capture of the language is relevant (e.g. for performing branching).
        3. @LANGUAGE-FORCE - When used on a field, the data entry form or survey on which the field is located will be rendered in the specified language (which must have been set up using the Multi-Language Management feature). The format must follow the pattern @LANGUAGE-FORCE="???", in which the ID of the desired language should be inside single or double quotes - e.g., @LANGUAGE-FORCE="de". Piping is supported - e.g., @LANGUAGE-FORCE="[field_name]". When the language is forced successfully (i.e., it exists and is active), the language selector is hidden. Using this together with @LANGUAGE-CURRENT-FORM/SURVEY on the source field for @LANGUAGE-FORCE may be used to 'lock in' a user to their selected language.
        4. @LANGUAGE-FORCE-FORM - Same as @LANGUAGE-FORCE, but the effect is limited to data entry forms (i.e. this does not affect surveys).
        5. @LANGUAGE-FORCE-SURVEY - Same as @LANGUAGE-FORCE, but the effect is limited to surveys (i.e. this does not affect data entry forms).
        6. @LANGUAGE-SET - When used on a Drop-down or Radio Button field only, this action tag will allow the field's value to control the currently shown language (in the same way as switching the language via the buttons at the top of the page). Tip: When used in a survey, this field could be prepopulated (and thus auto-selected) by embedding a participant's language ID in the survey URL itself (for details, see the FAQ's "How to pre-fill survey questions" section).
    • New feature: Form Display Logic
      • Form Display Logic is an advanced feature that provides a way to use conditional logic to disable specific data entry forms that are displayed on the Record Status Dashboard, Record Home Page, or the form list on the left-hand menu. You might think of it as 'form-level branching logic'. Form Display Logic can be very useful if you wish to prevent users from entering data on a specific form or event until certain conditions have been met. The forms will still be displayed on the page, but they will be disabled in order to prevent users from accessing them. Below you may define as many conditions as you want. A form may be selected in multiple conditions, but if so, please note that the form will be enabled if at least one of the conditions is met. The Form Display Logic does not impact data imports but only operates in the data entry user interface to enable/disable forms. Additionally, Form Display Logic is not utilized by the Survey Queue at all but can affect the behavior of the Survey Auto-Continue feature if the checkbox for it is enabled in the setup dialog. The Form Display Logic setup can be found by clicking the “Form Display Logic” button at the top of the instrument list in the Online Designer.
      • This feature serves as the official integration of the Form Render Skip Logic external module created by Philip Chase and his team. Thanks to them for their work on this module. Note: When upgrading REDCap to v12.0.0 or higher, if the Form Render Skip Logic is installed and is being used by any projects, all the configuration settings for the module will automatically be translated into the new Form Display Logic settings format, after which the external module will be disabled for each project and also for the entire system (since it will no longer be needed). This all happens automatically during the upgrade.
    • New feature: Design Checker for the Clinical Data Mart (CDM) The “Data Mart Design Checker” is a new tool available in the Data Mart fetch page that will report any issue related to the design of the current Data Mart project. Based on the most recent Data Mart XML template available in REDCap, the tool will check, list, and fix any of these issues: missing forms, variables, revisions, or section headers, the lack/presence of repeatability in a form, variables included in the wrong form, etc. An administrator or a user with Project Setup/Design privileges can use the tool to review and automatically fix all reported issues. This tool will mainly be utilized when users have modified the structure of an existing Data Mart project or if new forms and data types have been added to the Data Mart feature itself since the users initially created their Data Mart project.
    • Improvement: Errors displayed in the Survey Invitation Log when sending SMS or Voice Calls via Twilio will now display the full error message returned by Twilio's API to provide the user with more information regarding why the SMS/Voice Call failed to send successfully.
    • Major bug fix: When a field is embedded on a multi-page survey, in which the embedded field's parent field is used in branching logic on a later page, the embedded field's value might mistakenly get erased when a later survey page is submitted if the embedded field is set as a Required field. (Ticket #117620)
    • Bug fix: The cron job that sends email notifications for REDCap Messenger might mistakenly send multiple emails repeatedly to users. (Ticket #97084b)
    • Bug fix: The x-axis of a [scatter-plot] Smart Chart would mistakenly not display in the correct sorted fashion. (Ticket #117202b)
    • Bug fix: Clicking the "Today" or "Now" button for a date or datetime field, respectively, would mistakenly add the green highlighted background to the field if that field is embedded. Embedded fields should never get highlighted as green like regular fields do. (Ticket #105242)
    • Bug fix: When using the "Copy multiple fields" feature in the Online Designer, on some occasions the process might mistakenly fail for some fields selected and would display them on the page as fields with empty variable names. (Ticket #117339)
    • Change: The text for the "Example code" link at the bottom of the API Playground was modified for clarity. (Ticket #117797)
    • Bug fix: When using specific PHP versions, the Clinical Data Pull (CDP) service might mistakenly throw a fatal PHP error when attempting to fetch data from the EHR. (Ticket #117953)
    • Change: When drafted changes are auto-approved in a production project, the "Changes Were Made Automatically" dialog now provides extra text reminding the user that if any new instruments were just added, by default no users in the project have access to any newly created instruments. Thus they might need to grant users access to the new instruments.
    • Bug fix: When creating a new project or copying an existing one, the users that are initially granted access to the project would mistakenly not get logged as having been added to the project on the project logging page, thus making it very difficult for an auditor to determine exactly when and by whom the initial users had been given access.
    • Bug fix: A fatal PHP error would occur that prevented an administrator from creating a Data Mart project on behalf of a user. (Ticket #117929)
    • Bug fix: When using the Data Resolution Workflow in a project, the Resolve Issues page would mistakenly display data queries for fields that exist on instruments to which the user does not have data viewing privileges. (Ticket #118026)
    • Bug fix: If a value is piped into a Descriptive Text field which is itself embedded in another field, then in some specific instances the Descriptive Text field's label would mistakenly not get embedded but only the piped value would get embedded. (Ticket #117925)