How can I increase the speed of my data refreshes in Power BI using "Incremental Refresh"?

Get more timely data by reducing the time to refresh in Power BI using incremental refresh rates.

Refreshing your data in Power BI can sometimes take over 15 minutes based on the amount of data that is refreshed. You can reduce this time significantly by using Incremental Refresh rate. Incremental Refresh updates only the data that has changed since the last refresh within a specific date range, rather than reloading the entire dataset. Sound good? Here's how its done. 

Step 1: Create Data Parameters

Load a Weever form into Power BI and create Start and End date range parameters for the data. This is what the Incremental Refresh function will use to determine what to refresh and what to archive.

Step 2: Set up Date Fields to be Filtered by your Data Parameters

Set up the date fields in your data to be properly filtered by the Date Range parameters you created. 

Step 3: Enable Incremental Refresh

Enable the Incremental Refresh function to Archive old data and refresh recent data. 

 

Step 1: 

Create Data Parameters

Load a Weever form into Power BI and create Start and End date range parameters for the data. This is what the Incremental Refresh function will use to determine what to refresh and what to archive.

 

1.1:  Open Power BI Desktop, load your Weever data, then click on Transform Data. 
Screenshot 2024-08-07 at 1.51.39 PM

 

1.2:  Click on Manage Parameters under Home tab and then click on New Parameterimage-png-Aug-08-2024-07-25-13-5460-PM

 

1.3: Create 2 Parameters, one for the start of the date range you'd like the data to be refreshed, and one for the end of that date range. In this example these parameters are called ‘RangeStart’ & ‘RangeEnd’. 

  • For the Range Start Parameter add this information: 
    • Name:  Anything you’d like
    • Type:  Always ‘Date/Time’
    • Suggested Values: Always ‘Any Value’
    • Current Value for Range Start: Start Date from when you’d like to extract the data could be any number of years. Should be in YYYY-MM-DD HH:MM:SS AM/PM (ie. 2019-01-01 12:00:00 AM)
    • Tap OK
    • Tap "New" to make another parameter for the end range 
  • For the Range End Parameter add this information:
    • Name:  Anything you’d like
    • Type:  Always ‘Date/Time’
    • Suggested Values: Always ‘Any Value’
    • Current Value for Range End: End Date should be greater than today (Recommended: 2 years from today). Should be in same format as Range Start. Should be in YYYY-MM-DD HH:MM:SS AM/PM (ie. 2026-01-01 12:00:00 AM) 
    • Tap OK
    • Then tap Close & Apply

 

Step 2:

Set up Date Fields to be Filtered by your Data Parameters

Set up the date fields in your data to be properly filtered by the Date Range parameters you created. 

 

2.1:  Select your form from Queries Pane

 

2.2: Update the Data Type of the date columns (Form Created Date & Form Updated Date) to be in Date/Time  format. 

 

2.3: Click on filter arrow beside both 'Form Created Date' & 'Form Updated Date' columns, then tap Date/Time Filters, then tap Custom Filter (in most cases, it will be at the bottom of the list).

 

2.4: Complete the details exactly as shown in the image below. Click on the circled box and select Parameter instead of Date/Time which will be pre-selected.

 

2.5: Follow same steps for both the Form Created Date & Form Updated Date columns.

NOTE

Form Created Date = Date of Form Submission
Form Updated Date = Date the Submission was Updated (edited)

 

2.6: Once done, click on File / Close & Apply

 

Step 3:

Enable Incremental Refresh

Enable the Incremental Refresh function to Archive old data and refresh recent data. 

 

3.1:  Right click or tap the three dots next to your form name in the Data pane and tap Incremental Refresh. 

 

3.2: Select respective table and turn-on Incremental Refresh.

 

3.3: Enter number of years for which you’d like to see archived Data and for how many days you’d like to do incremental refresh.

  • Number of Years: 10 Years (Recommended)
  • Number of Incremental Refresh Days: 7 Days (Recommended)
  • Detect Data Changes: Turned on, choosing any column in your table EXCEPT 'Form Created Date' or 'Form Updated Date'.  

Number of Years & Number of Incremental Refresh Days : These settings will archive data from last 10 years and will only refresh the data based on either the Form Created Date & Form Updated Date within the last 7 days. This will speed up the refresh time and significantly reduce the load on our server as you will only be requesting required data and not all of the data during every scheduled refresh. Archived Data can be re-added anytime. Archived data won't be downloaded again from the server - only new data will be.

 

Detect data changes: This feature allows Power BI to detect the number of rows in the table to call out if changes have been made and if the data needs to be refreshed . This improves refresh performance, reduces resource usage, and ensures that only relevant data is processed. NOTE: Do not use 'Form Created Date' or 'Form Updated Date' columns for this feature. You can use any other column in your table, since it's just used for counting the rows. 

Archived Data: Yes it can be re-added anytime. Archived data won't be downloaded again from the server - only new data will be.

 

3.4: Click Apply and publish report to respective workspace and turn on Scheduled Refresh.

And that's it!

Now you know how to setup incremental refresh rates for reports!