Preparing a dataset for upload

Now that you have an account on Daylight and access to a Workspace, the next step is to upload some text data to analyze.

You will also want to include some metadata to provide context for the text data, such as demographic information, dates, scores, etc.

Metadata can be used to filter the data and analyze a subset of the dataset. Numerical data can also be used in Drivers analysis, which you will learn about later in this document.

Key vocabulary

Before we get started, here are some terms that we will be using throughout this section of the document.

• A verbatim is the conversational text component of the sample you have collected. 
• A document is a row of your source data, including the conversational text and any associated metadata.
• Metadata is structured data that creates context for text responses. Metadata may include demographics, dates, scores, or product details. 
• A CSV file, or comma-separated value file, is a plain-text file format that is used to organize data. CSV files exclude styling information that is included in an Excel XLS or XLSX file format. You can export a CSV file from most spreadsheet editors. 

From the Project List page, users with the Admins and Creator roles may create a new project with “+ New project” or add new data to an existing project with “+Add data”

The next page is where users identify the data to be analyzed in Daylight.

If you have a pre-formatted file, such as a survey export, you can either “Choose File” or drag and drop it into the top field.

Data Fields

The following table describes the types of data that can be included in the data to be uploaded. You may designate the data type for each column of the uploaded data during the uploading process. 

There are two ways to do this:

1. In your dataset file, you may include the data type as a prefix, followed by an underscore, and then the column name.  E.g. score_Rating, string_Location
2. As you upload your dataset file, you can change individual data types in Daylight.

Data type	Examples
Text (Required)	Column header: text or text_[FieldName]
The natural language verbatim for Daylight to analyze There must be one and only one Text column per file Each piece of Text may not exceed 500,000 characters in length	I loved the free coffee and the room was very clean, but it smelled strongly of cigarette smoke.I booked this room last-minute when my travel plans changed. The price was ok considering it was last-minute but it was way out of the way. We come to this hotel every year, and we appreciate the consistently top-notch experience!
Title	Column header: title or title_[FieldName]
A way to refer to each document is with the Title field. Title isn’t analyzed as part of the verbatim, but it can help organize documents. Only one Title column is permitted per data file. Titles are optional	Example: title Recent stay Hotel visit We’ll definitely be back!
String	Column header: string_[FieldName]
Words that help categorize the documents Can only filter fields in Daylight with up to 10,000 values Include as many string columns as needed	Example: string_MemberLevelNoneBusinessLoyalty
Number	Column header: number_[FieldName]
Numeric data  Can optionally use in the Driver feature Include as many number columns as needed	Example: number_Age184157
Score	Column header: score_[FieldName]
Score or rating data  Scores are numbers you want to be higher Recommended for using the Drivers function Include as many score columns as needed	Example: score_OverallExperience74.5 10
Date	Column header: date_[FieldName]
Date or times Accepts ISO 8601 strings, Unix timestamps, or US-style formats Daylight assumes that all dates are in a UTC timezone unless you include an ISO 8601 date with a specific timezone Helps you filter your project, especially if you upload data more than once Include as many date columns as needed	Example: date_CheckoutDate ISO 8601 formatted dates: 2026-04-10 2026-04-10T13:45 2026-04-10T13:45:00Z 2026 April 10 US-style dates: 04/10/2026 04/10/2026 13:45:15 4/10/16 1:45 PM April 10, 2026 Global Users: 10/4/26 will be inserted as October 4, 2016. If your dates have been uploaded incorrectly, please upload your data again and contact support@luminoso.com to adjust your document consumption counts for the duplicates.

Sometimes, a metadata field can have multiple values within a single document. For example, a survey may ask the respondent, “Which of these products have you tried?”. In such a case, the respondent may select more than one product. There are two ways that you can format the data in such cases:

1. Have one column for this metadata field and enter all of the values separated with the “|” (pipe) character. In the example above, the column header can be “Products Used” and the value in a given cell could be “ProductA | ProductB | ProductC”.
2. Have multiple columns with the same column header name with a single value in each cell. In the example above, you would have as many columns as you need with the header name “Products Used” and populate each cell with a single Product. Some of the cells can be left blank.

In both cases, a single metadata field will be created with multiple values.

Supported languages and multilingual datasets 

Daylight is capable of performing analysis natively in 14 languages. For best results with a multilingual dataset, split your data into one language per upload file.  Each language will be uploaded and analyzed as its own Project.  

Save as a CSV file

To upload your data to Daylight, save the file in CSV format, and make sure that the file extension is .csv. Daylight will also accept similarly formatted files, such as a tab-separated value (TSV) file. In this case, make sure that the file extension is .tsv. 

Uploading a dataset

To upload a file and create a new Project, go to the Project page and click on the New project button on the top right corner of the page. You can get to the Project page by clicking the Luminoso logo or “Luminoso Daylight” on the top left of your screen.

From this page, you can select a file to upload by clicking Choose file or by dragging and dropping a file onto the main panel on the page.

Note that you can only upload 1 file at a time.

Once a file has been selected, you will see a preview of your data, which will look similar to the image below.

Document Usage Meter

The document usage indicator at the top of the page shows document usage relative to the contracted document allotment for your organization. Before selecting a file to upload, you will see the licensed document amount and the currently used document count. Once a file has been selected, you will additionally see the number of documents that the selected file will use out of the allotment. 

Project Info section

The Project Info section consists of the Project name, Description, Workspace and Language. 

• Project name – defaults to the name of the uploaded file and can be edited
• Description (optional) – can be used for information about the project
• Workspace – defaults to whatever Workspace you were using when you clicked New project. If you have multiple Workspaces and prefer a different one, you can change it here. 
• Language – there is no default language selected. You must select one of the 15 available languages.

Data Summary panel

The Data Summary panel along the left side of the page is where you can designate the data type for each column of your data. There will be an element in this area for each unique column header name. If there are multiple columns with the same column header name, there will only be one element here corresponding to that group of columns. 

Use the drop-down at the top right of each element to designate the data type. A default value will be set as follows:

• If the column header is formatted as described in the Data Fields section, then that information will be used to set the default. For example, if one of the column headers is named “text”, the default for that column will be set to Text. Or if the column header says “date_ResponseDate”, the default for that column will be set to Date.
• Otherwise, the data type will default to String.

Each element will also provide information about the values found in that column in the context of the designated data type. For metadata columns, a preview of how this data will appear in the Filters after upload will be shown. 

Clicking on an element here will scroll the Data Preview panel to bring the corresponding column into view and highlight the column. 

Data Preview panel

The Data Preview panel shows up to the first 100 rows of the data. This can be used as a reminder of the contents of a column as you’re deciding on the data type designations. 

Clicking on a column in this area will scroll the Data Summary panel and highlight the corresponding element. 

Errors and Warnings

At the top of the Data Summary panel, a count of Errors and Warnings found in the data will be shown. Errors must be corrected before the project can be created. Warnings are things you should take a look at, but they will not prevent you from creating the project. The Create project button will not be enabled until all Errors have been corrected. The counts shown will change as Errors and Warnings are corrected or new ones introduced. 

Errors and Warnings that apply to the entire project will be shown just below the Errors and Warnings counts. They will be automatically dismissed as the issues are resolved.

Errors and Warnings specific to a column of data will be shown in the corresponding element in the Data Summary panel.

Note: Some errors and Warnings cannot be resolved within Daylight. In this case, you will have to make edits to your CSV file and upload the newer version.

Complete the upload

Once you have filled in the Project Info section, confirmed the data type designations, corrected all Errors and reviewed the Warnings, you can complete the upload process by clicking the Create project button on the bottom left. Just above the button, you will see a checkbox if you would like to receive an email when the analysis has been completed and the project is ready to be viewed.  It is checked by default.

Uploading to an existing project

The steps described in this section can be used to upload additional data to an existing project as well. To do this, click Upload data from either the Project’s Highlights page or within the Project tile on the Projects page. When there are mismatches between the data field names in the project and the column headers for the data being uploaded, Warnings will be shown to highlight these mismatches.