Data Processing

Processing survey data consists of four important steps:
Step 1: Data Entry
Capturing all the information from paper questionnaires and storing it in electronic format

Step 2: Data Editing and Quality Assurance
Editing and ensuring the maintenance of quality data

Step 3: Data Tabulation
Generating tabulations for both a key indicators and a final report

Step 4: Recoding of Datasets
Producing a clean set of data for use by researchers and policy makers

Step 1: Data Entry

This is the process of converting the information on the paper questionnaires to an electronic format. For the DHS surveys, this is done using CSPro, a software package designed and implemented by ICF Macro, the US Census Bureau, and others specifically to process survey and census data. CSPro is freely available from the US Census Bureau's website.

See Also:
Data Collection
DHS Survey Manuals

Step 2: Data Editing and Quality Assurance

One of the primary goals of The DHS Program is to produce high-quality data and make it available for analysis in a coherent and consistent form. Demographic surveys in developing countries are prone to incomplete or partial reporting of responses. Additionally, complex questionnaires inevitably allow scope for inconsistent responses to be recorded for different questions.  For the analyst this results in a data file containing incomplete or inconsistent data, complicating the analysis considerably.

In order to avoid these problems, The DHS Program has adopted a policy of editing and imputation which results in a data file that accurately reflects the population studied and may be readily used for analysis.

The quality of DHS data is assured by several processes:
  • Questionnaires are checked when they first arrive from the field, for the correct numbers of questionnaires and selection of eligible respondents. Responses that are open-ended (such as 'other' responses) or those that require coding (such as occupation) are also coded at this point.
  • All questionnaires are checked after data entry to ensure that all that were expected were in fact entered. The numbers of questionnaires are also checked against the sample design.
  • All questionnaires are entered twice and verified by comparing both data sets. All discrepancies are resolved.
  • The entered data are checked for inconsistencies and where possible, they are resolved. Some missing data, such as dates of events, are imputed where possible.
  • A set of quality control tables is generated on a regular basis. These tables indicate potential problems in the field. The tables include information on response rates, age displacement, and completeness of data. This information is then relayed to the field teams to help them improve the quality of data in the field.
See Also:
Data Quality and Use
Data Tools and Manuals

Step 3: Data Tabulation

Shortly after data entry is completed, the data processing specialist visits the implementing organization in the country again. During this visit, any additional data checking and cleaning is completed, and weights are calculated. This data set is referred to as the "raw" data. Data tabulation is done using CSPro, a software package designed and implemented by ICF International, the US Census Bureau, and others specifically to process survey and census data. There are two types of tables that are generated by the data processing team.

The first set consists of the Key Indicators Report tables. These tables are generated from the "raw" data set.  The number of the Key Indicators Report tables is limited and they present the main national key findings of the survey.

The second set consists of the Final Report tables. Production of tables for the Final Report can take several months to complete. The number of the Final Report tables is much larger than the Key Indicators Report and the data is presented in terms of national level statistics as well as for population subgroups and/or administrative or geographic subdivisions.

When appropriate to a topic, further data disaggregation is shown. The first step towards producing the Final Report tables is to generate a "standard recode" data set, which contains the same data as the raw data set, but in a standardized format. It is standardized in that the variable names and definitions are, wherever possible, consistent across all surveys. The "standard recode" is also important for researchers and policy makers since it produces a clean set of data for use. The second step is generating the actual Final Report tables. If possible, a set of the Key Indicators Report tables is generated in the time remaining during this country visit, while the complete set is generated at ICF International.

The Final Report tables are produced according to a set of standard tables, or a Tabulation Plan, which is established beforehand by the country manager and the survey specialists in country. The purpose of the Tabulation Plan is to provide model tables which set forth the major finding of the survey in manner that will be useful to policy makers and program managers. It also helps provide guidance concerning the most important indicators that should be presented in the survey report, the level of analysis expected and ensures timely dissemination of survey results.

See Also
Data Tabulation Plan

Step 4: Recoding of Datasets

The DHS Program makes the resulting survey datasets freely available to researchers, policy and decision makers. In order for the datasets to be clean and as comparable as possible across all surveys, The DHS Program generates "standard recode" datasets, which contain the same data as the raw datasets, but in a standardized format. In the "standard recode" datasets, the variable names and definitions are, wherever possible, consistent across all surveys. However, each survey is different, with questions that diverge from the standard. These questions are included in the standard recode datasets, either as computed standard variables or variables that are specific to that survey. The process of recoding can take several months and it involves consistency checking and comparisons between the standard recode and raw datasets.

Recoding of datasets is currently done for the DHS and AIS surveys. Work is currently in under way to recode the SPA datasets.

Recoding of DHS and AIS datasets
There are three core questionnaires in DHS surveys: the Household Questionnaire, the Women's Questionnaire, and the Men's questionnaire. There are also several standardized modules for countries with interest in other topics, such as malaria, domestic violence or maternal mortality. All additional modules are incorporated into the Household, Women's, or Men's questionnaires. There are two core questionnaires in the AIS surveys: the Household questionnaire and the Individual Questionnaire. The latter applies to women and men as well.

Since the survey methodology, sampling and eligibility of the DHS and AIS surveys are consistent, the DHS recode variables have been expanded to include the AIS variables as well.

Since the very beginning of DHS a recode file was designed for the sake of consistency and comparability across surveys. In the first phase of the DHS (DHS-I) the recode was defined only for the Women's Questionnaire. The recode file proved to be very useful and as a result since DHS-II, a recode file was introduced for the Household and the Men's questionnaires.

Recode files are initially created using a hierarchical model and later exported to flat files. There are two physical recode hierarchical data files. The first one includes the Household and Women's Questionnaire and the second one is for the Men's Questionnaire. The hierarchical data file is broken down into a number of records. The records were originally designed to map different sections of the model questionnaires, but because of changes among phases that is not the case anymore. Some of these records are repeating or multiple-occurrence records while others are single-occurrence records. Single records contain simple, single-answer variables. Multiple records are used to represent sets of questions that are repeated for a number of events.

There are special records to keep variables that are not part of the model questionnaires but were included in a particular country. These records are known as country-specific records and they can also be multiple or single depending on whether the question was added to a single or multiple section in the questionnaire.

See Also:
Data Tools and Manuals
Using Datasets for Analysis

Download Datasets

The DHS Program is authorized to distribute, at no cost, unrestricted survey data files for legitimate academic research. Registration is required for access to data.

Guide to Using Datasets


Debugging Information
ColdFusion Server Developer 2018,0,19,330149
Template /data/Data-Processing.cfm
Time Stamp 03-Jul-24 01:20 AM
Locale English (US)
User Agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
Remote IP 10.0.2.129
Host Name 10.0.2.129


Execution Time

Total Time Avg Time Count Template
703 ms 703 ms 1 C:/inetpub/wwwroot/commonspot/master.cfm
703 ms 703 ms 1 C:/inetpub/wwwroot/dhs/data/Data-Processing.cfm
672 ms 672 ms 1 C:/inetpub/wwwroot/commonspot/utilities/cached-page-render.cfm
485 ms 12 ms 40 C:/inetpub/wwwroot/commonspot/utilities/ct-render-element.cfm
438 ms 11 ms 40 C:/inetpub/wwwroot/commonspot/utilities/ct-render-element-core.cfm
344 ms 344 ms 1 C:/inetpub/wwwroot/commonspot/utilities/update-page-cache-status.cfm
313 ms 313 ms 1 C:/inetpub/wwwroot/commonspot/master-render.cfm
313 ms 313 ms 1 C:/inetpub/wwwroot/commonspot/render-page-meat.cfm
313 ms 313 ms 1 C:/inetpub/wwwroot/commonspot/render-page.cfm
313 ms 313 ms 1 C:/inetpub/wwwroot/commonspot/uncached-master.cfm
297 ms 297 ms 1 C:/inetpub/wwwroot/dhs/templates/template-basepage.cfm
265 ms 24 ms 11 C:/inetpub/wwwroot/commonspot/utilities/ct-render-named-element.cfm
63 ms 63 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c1378-l70-t51-dAny.read
47 ms 47 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c1433-l70-t51-dAny.read
32 ms 32 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c19155-l70-t51-dAny.read
31 ms 31 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c19154-l70-t51-dAny.read
31 ms 31 ms 1 C:/inetpub/wwwroot/commonspot/set-page-vars.cfm
16 ms 16 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c19153-l70-t51-dAny.read
16 ms 16 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c19172-l70-t51-dAny.read
16 ms 16 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c5191-l70-t51-dAny.read
16 ms 16 ms 1 C:/inetpub/wwwroot/commonspot/render-head.cfm
16 ms 16 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/public_api/livebookmarks.cfc | getTags(5297) ] from C:/inetpub/wwwroot/commonspot/public_api/livebookmarks.cfc
15 ms 15 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c19152-l70-t51-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c1432-l70-t51-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c19169-l70-t51-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c19174-l70-t51-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c19177-l70-t51-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c49703-l70-t27356-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c49731-l70-t27356-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c5184-l70-t51-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c5188-l70-t51-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c5193-l70-t51-dAny.read
0 ms 0 ms 1 C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c5195-l70-t51-dAny.read
0 ms 0 ms 1 C:/inetpub/wwwroot/ADF/core/load_site_adf.cfm
0 ms 0 ms 2 C:/inetpub/wwwroot/commonspot/controls/ct-include-once.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/controls/dhtmlmenu/menu-output-display.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/lview-contribute-mode-redirect-check.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/non-dashboard-include.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/pagemode/pagemodeui.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/render-body-tag.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/render-foot.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/render-metatags.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/render-tracking-code.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/security/security-page.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/site-application.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/startup/build-request-params.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/startup/core-application-meat.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/startup/handle-head-request.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/startup/handle-request-timeout.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/startup/init-customer-scope.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/startup/init-server-scope.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/startup/site-application-meat.cfm
0 ms 0 ms 17 C:/inetpub/wwwroot/commonspot/utilities/add-resource-request.cfm
0 ms 0 ms 6 C:/inetpub/wwwroot/commonspot/utilities/add-unregistered-resource-request.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/utilities/append-to-resource-group.cfm
0 ms 0 ms 3 C:/inetpub/wwwroot/commonspot/utilities/cp-cfheader.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/utilities/cp-cfmail-request-queue.cfm
0 ms 0 ms 2 C:/inetpub/wwwroot/commonspot/utilities/define-upfront-vars.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/utilities/flush-cache-db-queue.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/utilities/handle-in-list.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/utilities/load-template-properties.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/utilities/newwindow-js-meat.cfm
0 ms 0 ms 2 C:/inetpub/wwwroot/commonspot/utilities/newwindow-js.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/utilities/nocache-headers.cfm
0 ms 0 ms 2 C:/inetpub/wwwroot/commonspot/utilities/validate-date-string.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/commonspot/utilities/verify-stub-data.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/dhs/custom-application.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/dhs/custom-search/inc-custom-application-search-vars.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/dhs/data/Application.cfm
0 ms 0 ms 1 C:/inetpub/wwwroot/dhs/templates/template-basepage.head
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc | DBQueue_Count() ] from C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc
0 ms 0 ms 5 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc | GetPageCacheStatus(2352, 5, 2, 0) ] from C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc
0 ms 0 ms 2 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc | GetPageCacheStatus(5297, 5, 2352, 0) ] from C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc | ProcessDBQueue(noWait) ] from C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc | UpdateStatus([complex value], [complex value], 5, 5297, 2352, 0, 0, 90) ] from C:/inetpub/wwwroot/commonspot/components/cache/cache-status.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/custom-module-cache.cfc | getCustomAppPath(5) ] from C:/inetpub/wwwroot/commonspot/components/cache/custom-module-cache.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/custom-module-cache.cfc | getCustomModulePath(post-page-render, 5) ] from C:/inetpub/wwwroot/commonspot/components/cache/custom-module-cache.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/item-security-cache.cfc | GetEntry(5297) ] from C:/inetpub/wwwroot/commonspot/components/cache/item-security-cache.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/rewrite-rules-cache.cfc | getRewriteSettings(5) ] from C:/inetpub/wwwroot/commonspot/components/cache/rewrite-rules-cache.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/subsite-cache-manager.cfc | getLatest() ] from C:/inetpub/wwwroot/commonspot/components/cache/subsite-cache-manager.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/subsite-cache-manager.cfc | getSecurityPageIDMap([complex value]) ] from C:/inetpub/wwwroot/commonspot/components/cache/subsite-cache-manager.cfc
0 ms 0 ms 3 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/template-cache.cfc | GetTemplatePageData(2352) ] from C:/inetpub/wwwroot/commonspot/components/cache/template-cache.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/cache/template-cache.cfc | LoadTemplateProperties(template-Data-Template.cfm, 2352, 5) ] from C:/inetpub/wwwroot/commonspot/components/cache/template-cache.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/custom-application-framework/handle-applications.cfc | executeFunction(handleRequestStart) ] from C:/inetpub/wwwroot/commonspot/components/custom-application-framework/handle-applications.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/engine/ACF/utilities.cfc | checkFileExists(C:/commonspot-data/customers/commonspot-users/sites/commonspot-site-dhs/mapped/cache/5/p5297-c0-l90-t0-dlame.read, 0, false) ] from C:/inetpub/wwwroot/commonspot/components/engine/ACF/utilities.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/engine/ACF/utilities.cfc | checkFileExists(C:/inetpub/wwwroot/dhs/templates/template-basepage.cfm, 0, false) ] from C:/inetpub/wwwroot/commonspot/components/engine/ACF/utilities.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/commonspot-context.cfc | getDSN(site) ] from C:/inetpub/wwwroot/commonspot/components/util/commonspot-context.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent('head' request check) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(CGI variable processing) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(Request object updates) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(Request.Params initialized) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(RequestTimeout check) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(Server-level remote requests checked) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(Subsite context updated (ID=5)) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(authoring suspended check) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(basic setup) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(check for server property changes) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(core cleanup) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(core initialization) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(core variables) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(custom-application.cfm) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(customerScope object initialized) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(handleRequestStart) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(init-server-scope loaded) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(lock checks/updates) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(site environment initialization) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(site session check) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(subsite context refreshed) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(wait for lock: CommonSpot_Server_Scope (read)) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | addEvent(wait for lock: CommonSpot_Site1_Core (exclusive)) ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc | init() ] from C:/inetpub/wwwroot/commonspot/components/util/debug-timing.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/object-factory.cfc | getObject(Keywords) ] from C:/inetpub/wwwroot/commonspot/components/util/object-factory.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/object-factory.cfc | getObject(LiveBookmarks) ] from C:/inetpub/wwwroot/commonspot/components/util/object-factory.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/components/util/visitor-context-rule-actions.cfc | onPageVisit(5297, 5) ] from C:/inetpub/wwwroot/commonspot/components/util/visitor-context-rule-actions.cfc
0 ms 0 ms 1 CFC[ C:/inetpub/wwwroot/commonspot/public_api/keywords.cfc | appendEnforced(subsiteIDOrURL = , keywordList = ) ] from C:/inetpub/wwwroot/commonspot/public_api/keywords.cfc
31 ms  STARTUP, PARSING, COMPILING, LOADING, & SHUTDOWN
734 ms  TOTAL EXECUTION TIME
red = over 250 ms average execution time


SQL Queries

SelectedCustomRenderElements (Datasource=commonspot-site-dhs, Time=31ms, Records=2783) in C:\inetpub\wwwroot\commonspot\set-page-vars.cfm @ 01:20:49.049
		SELECT dcr.*
		  FROM Data_Custom_Render dcr, CustomElementModules cem
		 WHERE (dcr.PageID IN (?,?,?,?)) 
			AND dcr.VersionState=?
			AND (dcr.ModuleID = cem.ID OR dcr.ModuleID <= 0)
	 ORDER BY dcr.ControlID, dcr.PageID DESC, dcr.VersionID DESC
	
Query Parameter Value(s) -
Parameter #1(CF_SQL_INTEGER) = 1
Parameter #2(CF_SQL_INTEGER) = 2
Parameter #3(CF_SQL_INTEGER) = 5297
Parameter #4(CF_SQL_INTEGER) = 2352
Parameter #5(CF_SQL_INTEGER) = 2

Request.PageData.PageParams (Datasource=commonspot-site-dhs, Time=0ms, Records=1) in C:\inetpub\wwwroot\commonspot\render-page-meat.cfm @ 01:20:49.049
			SELECT *
			  FROM Param_Page
			 WHERE (PageID=5297 OR (PageID IN (2352,2,1) AND (VersionState = 2)))
			   AND VersionState=2
			 ORDER BY PageID DESC
		
qry (Datasource=commonspot-site-dhs, Time=16ms, Records=1) in C:\inetpub\wwwroot\commonspot\udf\validation\core.cfm @ 01:20:49.049
					SELECT SubsiteID
					  FROM SitePages
					 WHERE ID = ? 
Query Parameter Value(s) -
Parameter #1(CF_SQL_INTEGER) = 5297

testQuery (Datasource=commonspot-site-dhs, Time=0ms, Records=1) in C:\inetpub\wwwroot\commonspot\udf\validation\generated-validators.cfm @ 01:20:49.049
		SELECT ID
		  FROM SitePages
		 WHERE ID = ?
			AND (PageType IN (0,1) AND Uploaded=0)
	
Query Parameter Value(s) -
Parameter #1(CF_SQL_INTEGER) = 5297

qry (Datasource=commonspot-site-dhs, Time=0ms, Records=0) in C:\inetpub\wwwroot\commonspot\commands\livebookmarks.cfc @ 01:20:49.049
		SELECT Data_SeeAlso.ID, SubSites.SubSiteURL, SitePages.filename as pagename,
				 XMLPublications.name as feedname, XMLPublicationFormat.name as feedformat,
				 XMLPublications.description,
				 CASE
					 	WHEN XMLPublicationFormat.IsSyndication = 0 THEN 0
						ELSE 1
				 END AS IsSyndicationFeed
		  FROM Data_SeeAlso, XMLPublications, XMLPublicationFormat, SitePages, SubSites
		 WHERE Data_SeeAlso.PageID = ?
		 	AND Data_SeeAlso.Action = ?
		 	AND Data_SeeAlso.ItemID = XMLPublicationFormat.XMLPublicationFormatID
		 	AND Data_SeeAlso.TargetPageID = XMLPublications.XMLPublicationID
		 	AND XMLPublications.XMLPublicationFormatID = XMLPublicationFormat.XMLPublicationFormatID
		 	AND XMLPublications.PageID = SitePages.ID
		 	AND SitePages.SubSiteID = SubSites.ID
	
Query Parameter Value(s) -
Parameter #1(CF_SQL_INTEGER) = 5297
Parameter #2(CF_SQL_INTEGER) = 11

CacheUpdate (Datasource=commonspot-site-dhs, Time=0ms, Records=1) in C:\inetpub\wwwroot\commonspot\components\cache\cache-status.cfc @ 01:20:50.050
					UPDATE ElementInstance
						SET CacheFormat=?,
							 CacheValidUntil=?,
							 ContentValidUntil=?,
							 DataPageID=?,
							 DependentObjectID=?,
							 DynamicStatus=?,
							 HasChildren=?,
							 HasData=?,
							 HasDynamicContent=?,
							 IndexStatus=?,
							 IsDeviceDependent=?,
							 ParamPageID=?,
							 PrintStatus=?,
							 ShareStatus=?,
							 ConflictList=?
					 WHERE ServerID=?
						AND SubsiteID=?
						AND PageID=?
						AND ControlID=?
						AND CacheLevel=?
						AND IsWIP=? 
Query Parameter Value(s) -
Parameter #1(CF_SQL_INTEGER) = 0
Parameter #2(CF_SQL_VARCHAR) = 2124-07-03 01:20:49
Parameter #3(CF_SQL_VARCHAR) = 2124-07-03 01:20:49
Parameter #4(CF_SQL_INTEGER) = 0
Parameter #5(CF_SQL_INTEGER) = 0
Parameter #6(CF_SQL_VARCHAR) =
Parameter #7(CF_SQL_INTEGER) = 0
Parameter #8(CF_SQL_INTEGER) = 1
Parameter #9(CF_SQL_INTEGER) = 2
Parameter #10(CF_SQL_INTEGER) = 0
Parameter #11(CF_SQL_INTEGER) = 0
Parameter #12(CF_SQL_INTEGER) = 0
Parameter #13(CF_SQL_INTEGER) = 0
Parameter #14(CF_SQL_INTEGER) = 0
Parameter #15(CF_SQL_VARCHAR) =
Parameter #16(CF_SQL_INTEGER) = 1
Parameter #17(CF_SQL_INTEGER) = 34
Parameter #18(CF_SQL_INTEGER) = 20376
Parameter #19(CF_SQL_INTEGER) = 0
Parameter #20(CF_SQL_INTEGER) = 2
Parameter #21(CF_SQL_INTEGER) = 0


Scope Variables

Application Variables:
activesubsites=1,2,5,6,8,17,18,19,20,21,22,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58
adf=Struct (89)
adfscheduler=Struct (0)
applicationname=site1
assocpubdocs=Struct (4)
blogcache=Struct (4)
cachedimages=Struct (32)
cacheinfocache=Struct (53)
confidentiality=Array (5)
controlinstance_toplevel_cache=Struct (126)
covid_articlecount=10
covid_newsarticlecount=49
covid_pressreleasecount=4
covid_pubcount=3
cs_elementcache=Struct (10)
cs_resourcesmissing=Struct (0)
cs_rewriterulescache=Struct (21)
cs_subsitecachemanager=Struct (28)
customerinfo=Complex type
customerscope=Struct (21)
custommodulecache=Struct (21)
defaultconfidentiality=2
dhspagesyncdata=Struct (1)
itemsecuritycache=Struct (31)
languagenames=Struct (51)
languages=Struct (51)
lastknownchange=Struct (18)
lastupdate=Struct (14)
metadatacache=Struct (4)
pendingstubrequests=Struct (0)
rsscache=Struct (2)
search_archive_next_clean=2024-07-03 12:12:03
serverstatus=Struct (1)
site=Struct (74)
sitescope=Struct (9)
styleids=Struct (248)
stylenames=Struct (248)
subsitecache=Struct (46)
subsitecachebysecuritypageid=Struct (46)
templatecache=Struct (28)
templatedefaultsecuritycache=Struct (60)
templateenforcedsecuritycache=Struct (60)
treecache=Struct (25)
twittercache=Struct (2)
typefactory=Complex type
youtubecache=Struct (3)
CGI Variables:
AUTH_PASSWORD=
AUTH_TYPE=
AUTH_USER=
CERT_COOKIE=
CERT_FLAGS=
CERT_ISSUER=
CERT_KEYSIZE=
CERT_SECRETKEYSIZE=
CERT_SERIALNUMBER=
CERT_SERVER_ISSUER=
CERT_SERVER_SUBJECT=
CERT_SUBJECT=
CF_TEMPLATE_PATH=C:\inetpub\wwwroot\dhs\data\Data-Processing.cfm
CONTENT_LENGTH=0
CONTENT_TYPE=
CONTEXT_PATH=
GATEWAY_INTERFACE=CGI/1.1
HTTPS=off
HTTPS_KEYSIZE=
HTTPS_SECRETKEYSIZE=
HTTPS_SERVER_ISSUER=
HTTPS_SERVER_SUBJECT=
HTTP_ACCEPT=*/*
HTTP_ACCEPT_ENCODING=gzip, br, zstd, deflate
HTTP_ACCEPT_LANGUAGE=
HTTP_CONNECTION=
HTTP_COOKIE=JSESSIONID=A83681E0437E569F7722FE7A72464747.cfusion; CSRFTOKEN=069BC737A9602762364977F6059F4B367B574725
HTTP_HOST=preview.dhsprogram.com
HTTP_REFERER=
HTTP_URL=/data/Data-Processing.cfm
HTTP_USER_AGENT=Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
LOCAL_ADDR=10.0.0.111
PATH_INFO=
PATH_TRANSLATED=C:\inetpub\wwwroot\dhs\data\Data-Processing.cfm
QUERY_STRING=
REMOTE_ADDR=10.0.2.129
REMOTE_HOST=10.0.2.129
REMOTE_USER=
REQUEST_METHOD=GET
SCRIPT_NAME=/data/Data-Processing.cfm
SERVER_NAME=preview.dhsprogram.com
SERVER_PORT=80
SERVER_PORT_SECURE=0
SERVER_PROTOCOL=HTTP/1.1
SERVER_SOFTWARE=Microsoft-IIS/10.0
WEB_SERVER_API=
Cookie Variables:
CSRFTOKEN=069BC737A9602762364977F6059F4B367B574725
JSESSIONID=A83681E0437E569F7722FE7A72464747.cfusion
Request Parameters:
anonnoreadelements=
applockname=CS_App_Site1_1
archiveemail=archive@dhsprogram.com
assetsdir=/assets/
author_approve_whereclause=(VersionState = 2)
authormode=0
authorok=1
blupfront=
browser=Struct (13)
browserallincluded=1
cachefilesused=39
cachestatus=Complex type
cfflushcount=0
cgivars=Struct (49)
cloudfrontkeypairid=K3LSHNFSZFMIUC
cloudfronturl=d1vyaiefsqy4pf
constants=Complex type
cors.isCorsRequest=NO
cp=Struct (123)
cs_includedobjects=nwjs
cs_init_timings=Struct (23)
cs_isdashboardcmd=0
cs_request_start_memory=1222685552
cs_runningdashboardcmdcode=0
csrftoken=069BC737A9602762364977F6059F4B367B574725
customcfupfront=
customerinfo=Complex type
customhead=1
customrenderhandlerpairs=
customsearch=Struct (11)
dbdsn=measure_db
debugstate=Struct (1)
dhspagesyncdata=Struct (1)
dhtmlmenuitemsprefetch=
disablehideformenus=0
dsource=dhs_legacy_live
editmode=0
element=Struct (42)
elementrundetail=Struct (0)
elementsrun=0
environment=Struct (5)
finalreportids=5,26,21,10,18,30,39
finalreportna=139,315,445,488,107,138,459,518
formattedtimestamp=2024-07-03 01:20:49
getghost=
hivbold=1,3,8,9
imageupfront=
imgdir=/images/
includerootdir=/
initialsiteid=1
initticks=1719969649691
jswarnmessage=
linkdataupfront_itempos=
linkdataupfront_itemtitle=
lockid=0
mainpage=Struct (84)
master_cfm_used=1
maxperms=Complex type
modeorderbyclause=VersionID Desc
modewhereclause=(VersionState = 2)
modewhereclause_page=(VersionState = 2)
multisectionupfront=
noauthorreason=
nocacheelementtypes=
nocacheheaderincluded=1
nondashboardincludedone=1
obcountires=EG,ZM
package=Struct (0)
page=Struct (84)
pagedata=Struct (1)
pagedhtmlitems=
pagedhtmlmenus=
pageperms=Complex type
pagesync=Struct (4)
params=Struct (4)
params_formfields=
params_urlfields=
readmodewhereclause=(VersionState = 2)
renderedelements=Struct (0)
renderelementdata=Struct (0)
renderfordiff=0
renderforprint=0
renderstate=Struct (73)
req_cam_callcount=1
req_inittimestamp=2024-07-03 01:20:49.691
req_sequenceid=56144
req_startticks=1719969649691
rootdir=/
rootpath=
rooturldatasetadmin=https://preview.dhsprogram.com/data/dataset_admin
samesite_value=
scheduleupfront=
serverdatasource=commonspot-sites
sessionlockname=Session_1_A83681E0437E569F7722FE7A72464747.cfusion
sessionstring=1_A83681E0437E569F7722FE7A72464747.cfusion
sicupfront=
sicupfrontparam=
site=Struct (74)
siteid=1
standardrenderhandlers=
styleids=Struct (248)
stylenames=Struct (248)
subsite=Struct (0)
subsitecache=Struct (46)
subsitecachebysecuritypageid=Struct (46)
subsiteid=5
taiupfrontparam=
tbupfront=
tbupfrontparam=
templatecachestatus=Struct (1)
templaterootdir=/
timestamp={ts '2024-07-03 01:20:49'}
trustcachestatus=1
typefactory=Complex type
udslockname=CS_Customer_commonspot-users
updatingschemas=0
user=Struct (30)
usersdatasource=commonspot-users
wanterrorsrethrown=0
warnmessage=
xmlbrowser=Struct (13)
Server Variables:
ADF=Struct (15)
COMMONSPOT=Struct (44)
EMAILALERTKEY=BSu0cRa8xSWdSrClnnGvpA==
coldfusion=Struct (10)
os=Struct (5)
system=Struct (2)
Session Variables:
_csrf_token=069BC737A9602762364977F6059F4B367B574725
adf=Struct (0)
authormode=0
contributemode=read
currentpageid=5297
debugmode=0
debugstate=Struct (1)
editmode=0
environment=Struct (5)
groupbyregion=
hasusertempfiles=0
language=Struct (216)
lastupdate=Struct (2)
lastuserupdate=1719969648098
lockid=0
lockobjectid=0
lockobjecttype=
locktype=
logintime=
ownedobjecttypes=
pendingapprovalpages=
pendingapprovals=
previewmode=0
requestedversioncounter=
requestedversiontimestamp=
sessionid=A83681E0437E569F7722FE7A72464747.cfusion
sessionstring=1_A83681E0437E569F7722FE7A72464747.cfusion
urltoken=CFID=2271390&CFTOKEN=f596d97df9c9c6bd-9DCC5E95-C6C7-BF81-26D18BCD005CDE3E&jsessionid=A83681E0437E569F7722FE7A72464747.cfusion
usagelist=
user=Struct (30)
xmlbrowser=Struct (13)
xmlenvironment=Struct (5)
Debug Rendering Time: 31 ms