Sierra Charts Backtester (PssSCBT) Main Page
"Automatically run backtests for your Sierra Chart Trading Strategy!"

Welcome to the new "Sierra Charts Backtester (PssSCBT)" which now actually allows you to "remote control" Sierra Charts and 
run many backtest permutations automatically - using parameter ranges you define for each backtest run! Check out this Video Introduction and Demo Videos
  
The new "Sierra Charts Backtester (PssSCBT)" and the "Backtesting & Walk Forward Manager (BTWFMgr)" is listed on the Sierra Chart 3rd Party Link Page:
Custom Sierra Chart Development: We are also available to develop custom Sierra Charts strategies/indicators/consulting etc for you - email to pss@pobox.com
 
Running a Backtest:
View Permutations: 
When you open SCBT - you see all input ranges for "RSI" the selected backtest definition (BTD)
In this "RSI" example - we have in total 624 permutations, using the three Parameter: RSILength(13x) Overbought(8x), Oversold(6x):
 
To view the actual Permuttaions - click on the "ShowPerm" button - which will create a spreadsheet with all combinations in the "C:/BTWFMgr/SCBT/Perms.csv" file
 
Prepare Backtest: 
Before you start the automated backtest - open the Trade Activity Log - Trade/Trade Activity Log


and make sure you the "Trades" TAB is selected:
Start Backtest: 
To start the automated backtest - simply click on the "Start Sierra Chart Remote Backtest" button! and confirm the run:

Which will then start to run though all the 624 permutations 
It is best to leave the computer alone during the backtest
as other activities can interfere with the monitoring and remote controlling of Sierra Charts from SCBT!


SCBT will also show the current speed of completed permutations per hour (114.3) and etsimated time of completion ETA: 18:10:18

Pause/Abort Backtest: 
To pause the automated backtest - simply click on the "Pause" button! - which will change to "Continue" and SCBT will wait for you to click the "Continue" button
To abort the automated backtest - simply click on the "Abort" button!
Backtest Completion:
When all the permutations have been processed the completion confirmation box is displayed:

and the main SCBT windows will show the processing speed etc - so you can better estimate the next backtest run etc:


When the backtest completes - SCBT will:
SCBT is creating the Summary.csv spreadsheet window (Summary.xls)  - with detailed statistics of All , Long and Short Trades:
:
 
Open the "Backtesting with Walk Forward Manager (BTWFMgr)"  with the collected BTWFMgr.btwf1+2 data:

 
Open the Trades/Equity Performance Chart for the best TotalPL results - with the "Trading Performance Analyzer (TrdPerfAna)PermXXXX.posa  
 
Open the backtest folder with the regular Windows Explorer, so you can view the files and open any result file you would like:
 

 

Backtest Statistics:
SCBT is creating the Summary.csv spreadsheet window (Summary.xls)  - with detailed statistics of All , Long and Short Trades:
:
 
You can also any of the Permutation results with the "Trading Performance Analyzer (TrdPerfAna)" and open very detailed the Statistics (File/Show Statistics):


Backtest Speed:
The backtest steps for each permutation - as outlined in the "Internal Backtesting Steps" below - take around 26 Seconds per permutation and depend on:
a) computer speed
b) number of bars in the chart (MSFT_RSISample  has approx. 3000 bars)
c) complexity of the trading DLL logic


You can enhance the backtetsing speed to some extend by increasing the "Bar Processing Increment" from the default 250 bars::

The "Bar Processing Increment" specifies the number of bars that are processed at once before user interface input is processed.

to  for example 1000 bars:


which increased the speed from 140 permutations per Hour(26 Seconds)  to 232 permutations per hour (15 Seconds)

To adjust the SCBT "Bar Processing Increment" - open the Configuration: File/Configuration

click on the "BarIncr" item and enter for example "1000" as the new value and click on APPLY and confirm the change:
 
On the next backtest SCBT will then use 1000 Bars instead of the default 250.

BarIncr Maximum 1000
It seems that after than 1000 Bars - do NOT affect the backtesting speed in Sierra Charts?

Creating/Starting a New Backtest Definition(BTD):
To create a NEW backtest definition(BTD) - simply select "{New}" from the Pull down list:

OR: Select in the File menu: File/New Backtest Definition

Then select the Sierra Charts Chartbook you owuld like to use for the new backtest:


Then enter the new backtest definition name:


SCBT will open the selected Chartbook, extract the numeric Inputs from the Indicator(s) and display the list for you to edit:

Initially all increments are set to 0 and the Min/Max values are identical, so no permutations result.

Now edit the Inputs - simply double click on the cell and enter the new value and ENTER - for example RSILength 6 to 30 in increments of 2:


As you update the values - the backtest definition (BTD) file is also updated in: 
C:/BTWFMgr/SCBT/Backtest1.btd


You can also manually edit your Backtest Definition(s)

Manual Backtest Definition Editing:
You can also manually edit your Backtest Definition(s) - simply select the Backtest Definition from the list and click on the "Edit" Button:

Sierra Chart from 3rd Party(Brokers etc):
Some Brokers make Sierra Charts available for their clients - see list here - but use a special Sierra Charts folder and a branded Sierra Charts release.
Example: InfinityFutures and InfinityAT Charts from TransActFutures runs SierraChart.exe from their special subfolder: C:/SierraChart/SierraChartTransActMA/SierraChart.exe 
instead of the regular: C:/SierraChart/SierraChart.exe 

SCBT allows you to switch the Base folder to a different location: 
Open: File/Configuration - then change the "Base" Setting to:: C:/SierraChart/SierraChartTransActMA and click on "Apply"
 

 

Consistency Between Back Tests/Backtesting Replay Modes:
Select the "Replay mode" to be used for your backtest in the SCBT pull down menu:
  

Sierra Charts recommends the "Slow - Time based mode(Time based - Accurate Trading System Back Test Mode)":
If you are performing a replay based back test of an automated trading system when using the Spreadsheet System for Trading study or an 
Advanced Custom Study that uses the ACSIL Trading functions, then you need to set the Replay Mode to Accurate Trading System Back Test Mode.

Consistency Between Back Tests
If your trading system gives inconsistent results between Back Tests without making any changes to the chart data, 
the Chart Settings or to Study Settings, you must look at your own trading system code/formulas for a cause. 


Which corresponds to the 3 different Replay Modes Sierra Charts offers :
Fast Bar Based Replay
Medium Time Based - Standard Replay:
Slow Time based - Accurate Trading System Back Test Mode

 

Backtest Data Collection:
During the backtest SCBT will automatically collect all the data for each backtest permutation:
The Sierra Charts Trade Logs for each permutations and save in the PermXXXX.txt files
Convert the Sierra Charts Trade Logs into the "Trading Performance Analyzer (TrdPerfAna)PermXXXX.posa for visual trades display as charts and advanced analysis:
Perm00338.posa - Show the equity chart all 70 trades with the blue income line: Perm00338.posa - Show all 70 trades separated by weekday - Monday/Tuesday are best!
Show the the backtest Summary spreadsheet (Summary.csv)- displaying main statistics (Win Rate%, TotalPL, Trades etc) for  each backtest permutation:
  
Collect and display the data for the "Backtesting with Walk Forward Manager (BTWFMgr)"  
so you can actually view and analyse ALL the backtesting results in much more depth and run Walk Forward analysis etc.
Note that BTWFMgr is a separate powerful product and not part of SCBT! 
Location: C:/BTWFMgr/SCBT/Backtests/RSISample/SEQ_00XX/BTWFMgr.btwf1+2
Example: Equity Distribution overview for the 624 Permutations:

Effect of the RSILength input parameter - on the overall  backtest results:

and the effect of the RSILength on the Number of Trades - as expected with longer RSILength - less trades:
Effect of the Overbought input parameter - on the overall  backtest results
 

 

File Locations:
The SCBT base folder is: C:/BTWFMgr/SCBT   
The SCBT backtest definition file: C:/BTWFMgr/SCBT/*.btd     
The SCBT backtest log files are in: C:/BTWFMgr/SCBT/logs/YYYYMMDD/scbt.log     
The SCBT backtest results folder is: C:/BTWFMgr/SCBT/Backtests/{StratName}RSISample/SEQ_00XX  (XX is the backtest counter, {StratName}is the Strategy name i.e. RSISample)
The SCBT backtest summary spreadsheet: {backtest results folder}/Summary.csv
The SCBT backtest BTWFMgr data: { backtest results folder}/BTWFmgr.btwf1+2
The SCBT backtest Permutation Trading result: { backtest results folder}/Perm00XXXX.posa+txt

The RSI Sample Trading DLL module is in: C:/SierraChart/Data/RSISample.dll     
The RSI Sample Trading chartbook is in: C:/SierraChart/Data/MSFT_Chart_RSI.cht     
The RSI Sample Trading data is in:  C:/SierraChart/Data/MSFT.csv     
The RSI Sample Trading module C++ code is in: C:/SierraChart/ACS_Source/RSISample.cpp
  
 
Internal Backtesting Steps:
The SCBT module performs a lot of internal steps in order to run reliably the remote controlled backtests:
Part1 - Preparation:
Make sure Sierra Charts is opened
Extract the Strategy name from the "Backtest Definition (BTD)" file
Close and "left over" Sierra Charts windows 
Open only the chartbook selected for the backtest - close all other chartbooks 
Disconnect from the live data - for the simulated backtest
 
Create the next available Backtest run folder:
C:/BTWFMgr/SCBT/Backtests/{StratName}RSISample/SEQ_00XX    - XX is the backtest sequence counter and  {StratName} is the Strategy name like RSISample 
Copy backups into the Backtest run folder of the a) "Backtest Definition (BTD)" file b) Chartbook file c) Config file (SCBT.ini)
Load all the available Indicators and their associated Inputs
Calculate the number of permutations (624)
Confirm that we start the backtest:
 

Part2 - Each backtest permutation:
Adjust all Strategy input values to the next permutation
Check for pause/abort
Close and "left over" Sierra Charts windows 
Imprint the new Strategy input values from this permutation into the chart
Run the replay and wait for the replay/backtest to complete
Save the backtest results into the Backtest run folder:
- Open the Trading Activity Log
- Save the trades as "C:/SierraChart/TradeActivityLogs/SCBT.txt"
- Close the Trading Activity Log
- Move the "C:/SierraChart/TradeActivityLogs/SCBT.txt" Sierra Charts Trade Log for each permutations and to the PermXXXX.txt files
- Convert the Sierra Charts Trade Logs into the "Trading Performance Analyzer (TrdPerfAna)PermXXXX.posa 
- Collect the new backtest trading data for the "Backtesting with Walk Forward Manager (BTWFMgr)"  
- update the backtest Summary spreadsheet (Summary.csv)
 

Part3 - When backtest completed
Save last trades data for the "Backtesting with Walk Forward Manager (BTWFMgr)"  
View the saved backtest data with "Backtesting with Walk Forward Manager (BTWFMgr)"  
Location: C:/BTWFMgr/SCBT/Backtests/RSISample/SEQ_00XX/BTWFMgr.btwf1+2
View the backtest Summary spreadsheet (Summary.csv)
Show the completion box: