Diamond Backtesting
with
Walk Forward Manager
(BTWFMgr)
the
new cutting edge in Backtesting
sharp and clear like a diamond
by
Professional
Software Solutions
(Burkhard
Eichberger)
http://www.ProfSoftware.com/bt
Copyright ã
20042017 by Burkhard Eichberger. All rights reserved.
No part of this book may be reproduced in any form or by any electronic or
mechanical means including information storage and retrieval systems
without permission in writing from the author (pss@poboxm)
(BTWFMgr Version 3.X)
Table
of Contents
Fully automated advanced Walk
Forward Analysis (WFA)
Innovative Volatility Analysis
Advanced Strategy Input Parameter
Analysis
Position Profit/Loss Distribution
Analysis
Innovative Trading Strategy
Potential Analysis and Optimization
Identify the best Weekdays and
Entry Intraday times
Detect the Market Long/Short Bias
Detect the Strategy Trading
Frequency
Find the best exit logic  Position
Exit Manager (PEM):
Visual Multi Timeframe/Symbol
Analysis  Result Overview Module
Integrated In depth Performance
Analyzer (PerfAna)
Export Results to Microsoft Access
Database
Correlate Context Values with best
Potential
Automated modification of your
EasyLanguage Strategy Code
Equity/Walk Forward Mode
(nBTWFMgrExport=2)
Potential Mode (nBTWFMgrExport=1)
Data Collection turned off
(nBTWFMgrExport=0)
BTWFMgr Configuration and
Preferences
Step1 – Strategy Preparation &
TradeStation® Optimization
Step2 – Detecting the optimal
Strategy Potential/Parameters
Step3 – Detecting the optimal
Strategy Exit Method
Step4 – Detecting the optimal
Strategy Result
Step5 – Walk Forward
Analysis/Optimization (WFA)
Walk Forward Analysis Overview
Walk Forward Analysis Setup Module
Walk Forward Analysis – Filter
Definition (Step1)
Walk Forward Analysis – Sort
Definition (Step2)
Walk Forward Analysis – Defining
the Period Setup/Sequence
Defining a sequence of walk forward
Period setups
Walk Forward Analysis – Check your
Save Result Options
Walk Forward Analysis – Starting
Walk Forward Analysis
Walk Forward Analysis – Treeview
Display
View best WFA Drawdown, Volatility,
ProfitFactor and Robustness
Preparing TradeStation®  Opening
your Strategy Code
Adding the BTWFMgr Data Collection
Section
Sample of Data Collection Section
Detecting the optimal Strategy
Potential/Parameters
Analyzing the Strategy Per Trade
Potential
Analyzing the Strategy Yearly
Potential
Analyzing the Strategy Parameter
Potentials
Analyzing the Strategy Context
Variables
Detecting the optimal Strategy Exit
Method
Analyzing the Strategy Exit Results
Analyzing the Strategy Exit
Parameters
BTWFMgr Configuration and Preferences
Smart Ranking Analysis Settings
Best Trade Permutation Analysis
Settings
Walk Forward Analysis+Optimization
Settings
Strategy Potential &
Probability Analysis Settings
Initial Data Conversion Settings
Position Exit Manager (PEM)
Settings
Developing
a successful trading strategy is a craft  some might even say an art.
You
will need the right tools to be able to test, evaluate and optimize your
trading strategies.
To
assist you in this task and adventure  we have developed a new powerful
software tool:
”Diamond Backtesting with Walk Forward Manager”
– or in short BTWFMgr.
For more current information and to download go to http://www.ProfSoftware.com/bt
We have grouped the BTWFMgr manual into the following chapters:
·
Overview
Presents the main BTWFMgr functions with a brief description
·
BTWFMgr Basics
Introduces you to some basic concepts of BTWFMgr and
shows you how to get started and some basic BTWFMgr functions
·
Strategy Development
Describes some basic concepts how strategies can be developed (using BTWFMgr)
·
Function Reference
Here we show you each BTWFMgr function in greater detail
·
Licensing & Ordering
Details
about ordering BTWFMgr and licening
·
Installation
Stepbystep
guide how to install BTWFMgr
·
Glossary
Explains
basic terms and abbreviations used in this BTWFMgr manual
We
hope you will find BTWFMgr a valuable tool and invite your feedback at: pss@pobox.com
Check
frequently for updates – as we continuously improve and expand BTWFMgr at:
www.profsoftware.com/bt/dl.htm
BTWFMgr works for example with the TradeStation Open Platform® (TradeStation.com)
and enhances one of the most powerful trading and strategy research
platforms available
which seamlessly integrate the charts and strategy and fully automated
executions.
Below
we will introduce you to the main functions BTWFMgr is offering you,
so
you can analyze and improve your strategies.
For more details on each function use the Reference Chapter
Provides
you with a powerful, more realistic backtesting method, testing essentially the
predictability/robustness, instead of the usual unrealistic curve fitting
optimization,
saving you hours or even days of cumbersome calculations. Below is a 4
week/1week example:
All
aspects of the walk forward analysis can be customized:
Filter, Sort, In/OutSample Sequences/Clusters  using intuitive Filter/Sort and
Period Manager interfaces.
Walk
Forward Analysis Results can be shown even in a 3D cluster view
presenting in this example the correlations between in and out sample
variations:
Also
results can be viewed as a spreadsheet/2D view.
Below is an example showing the clusters (using 5 to 30 runs and 10 to 30% out
of sample):
These
Walk forward result views can use any of the many statistics available:
Finds
you the best backtesting results by combining your most important
(customizable) sort criteria into a hybrid. Below is an example using
Equity(3x), Volaitility(2x) and Drawdown(2x):
The ideal result would be at the top in all sort criterias = 100%,
below is an example, showing the smartranbking, with the best hybrid result
(99.44%) on top:
For
most traders a fairly predictable, smooth equity graph is preferable to
an equity graph with wider swings, but a higher ending equity. The innovative
volatility analysis measures the “swing” around the smooth equity income line.
Below is an example of the fairly smooth equity graph:
below
we see the same ending equity($5400) but a much higher volatility(17%) and
wider “swings”:
Shows
you a detailed visual analysis of the effect of EACH strategy input parameter
variations on the overall result, below is an example in 3D view, combining RSI
oversold and overbought parameter:
Also
a 2D graph is available  showing the overall(blue),
long(green) and short(red)
effect of each parameter value on the overall result:
Helps
you detecting misleading outliers in your equity graph:
ranking winners(green) and losers(red) – and showing the overall average line at
$240:
Track
how the market response on average to your entries (Potential Analysis),
helping
you to detect the maximum trading opportunities and finding best entry points,
using
ONLY ENTRY POINTS, before you focus on develop the exit logic in your strategy.
Below is an example of a stable gain:
and
an example of a gain which expires after around 45 bars since entry:
Identify
the best intraday trading times of your startegies.
In the “Equity Time Analysis” sample below
we see that 9:43 to 10:13 and 14:13 to 15:43 are the best trading times:
(You
can adjust the time interval in the configuration: Treeview/EquTimeLength  the
default is 30 minutes)
This
function allows you identify the best weekday to trade  for all(blue), long(green)
or short(red).
In
the sample below we can see that Monday is the best trading day:
This
function allows you to detect the general long(green)
or short(red) bias (if any)
of your strategy performance. Also periods of strategy sub performances are
shown:
You
can also use a more detailed weekly market long/short bias analysis:
This
function allows you to detect how frequently your strategy is trading over
time,
separated by long(green), short(red) and total
trades(blue):
You
can also see the more detailed weekly (and even daily) frequency analysis
After
you have identified the best strategy entries with the best potential, you can
test millions of exit logic variations, using the integrated high speed
(100,000/second) Position Exit Manager (PEM),
to detect the best way to exit. You can use any combination of various exit
functions:
Exponential+Linear
Target, Exponential+Linear StopOut, Trailing, breakeven, age etc.
Below we see a visual example how PEM can manage your positions:
The
"probability map" function, allows you to detect the best probability
areas,
in the time based market response (Potential) analysis.
Below we can see an the best probability is reached after 75 bars,
for the three (customizable) target levels: $50(green),
$100(magenta), $150(blue):
After
you have run several backtests for a strategy across symbols and/or timeframes,
you
can quickly detect the best performance using Probability%, Yearly, PerTrdAvg
or PeakAvg$,
calculated
separately by long/short or merged trading,
below
is a screen shot with the best probability results for shorts:
In
the chart three lines are shown:
·
green
thin line = single top result
·
blue
thick line = average of the top 10 results
·
blue
think line = avgerge of the top 100 results
This
additional module allows you to perform many in depth equity result operations
and statistics
and also export the trades to a spreadsheet – below is a statistic analysis
view:
You
can merge results from different markets/symbols into one new comprehensive
result/graph
with the Performance Analyzer module.
This
function allows you export the permutation data to a standard microsoft access
database.
Then you can apply any advanced SQL queries and/or reports to process and
detect the best results:
(just
enable the export by setting Setting System/ExportTrdPer to YES and the next
time you open an equity result  all permutations will be exported)
You
can add any variable in Potential Mode to the backtesting, so you can detect
any correlation between context variable range(s) and favorable potential
results.
In the example below, we use the angle exponential average (see PSS_RSISample
workspace),
detecting that slightly downward angle (around minus 0.5264) offer the best
returns for shorts(red):
Provides
you  with a mouse click  with the actual charts of each Position Entry, Exit
and Risk/Reward excursion instantly, so you can evaluate also visually all
entries and their performance  instead of using the cumbersome scrolling in
the charts – in the example below we see:
a) the horizontal blue line (entry price)
b) maximum favorable excursion (green line)
c) maximum unfavorable excursion (red line)
d) Exit price (end of diagonal blue line):
This
function completely automates the preparation of your strategy EasyLanguage
code,
so BTWFMgr can collect all the trades data during the optimization.
(you don't have to be familiar at all with EasyLanguage!)
Below is the interface through which you can modify your strategy code
(PSS_RSISample):
BTWFMgr
is using a very visual approach, which (hopefully) makes working with BTWFMgr easy+intuitive.
Similar to the windows explorer, all relevant items are shown in a window on
the left side (treeview),
were can “zoom” in or out any item. The “display”
shows the current details as a chart or even 3D View.
Main Display (1) 
shows
the chart, list or 3D view for the current item from the treeview 
Treeview Area (2) 
Shows
the items for the current backtest data in “branches” and subbranches. 
Popup Menu(3) 
The
list of specific functions for the currently selected element/item. 
Toolbar(4) 
List
of the main functions – using icons  accessible via one mouse click 
Menu(5) 
All
available functions  accessible via the standard Windows menu interface 
Tooltip window(6) 
A
small white window popping up when you move the cursor in the main display 
BTWFMgr operates in two main modes:
a) Equity/Walk
Forward Mode: entry and exit points are used for an equity graph for each
permutation
b) Potential
Mode: only entry points are used to detect the best market response for each
permutation
When running the optimization, simply
set the “nBTWFMgrExport” strategy parameter to 2(Equ) or 1(Pot).
See “Strategy Preparation” for
details how to prepare your strategy for BTWFMgr.
This is the main mode, were the strategy generates entries AND exits automatically, applying the “Strategy Entry+Exit Logic” for trading decisions. In this “Equity(Equ)” mode, we are optimizing the Equity and overall Strategy results and can run the advanced walk forward analysis, testing predictiveness.
BTWFMgr comes with an equity sample
“out of the box”, so you instantly can test drive BTWFMgr:
Start/Programs/Diamond
Backtesting/Sample Equity + Walk Forward Analysis
Below are the main Treeview branches
showing the results (more details in further chapters):
Title 

at the very top showing the symbol,
time frame and the description. In this example: 
Walk Forward Optimization
Results 

Showing the top 10 Walk Forward
results; Equity total. attached under “134 Other
Optimizations” 
Strategy Overall 

Showing the overall results or each
permutation. ·
Sorted by Max Equity ·
Sorted by Average Drawdown (%) ·
Sorted by PSSVolatility (%) ·
Sorted by Probability (%) ·
Sorted by ProfitFactor ·
Sorted by SmartRanking1
(Custom Hybrid1) ·
Sorted by SmartRanking2
(Custom Hybrid2) ·
Sorted by SmartRanking3
(Custom Hybrid3) ·
Sorted Permutation Sequence
Number Here you can find out the best
overall results. 
Strategy Input Parameter Analysis 

In depth Analysis of strategy input
parameter: b) each
individual input parameter c) each
input parameter value d) even
their respective sub values Here you can detect the optimal
parameter range. 
Context Parameter Analysis 
In this section we track the overall
effect of each 
This is the initial mode in your
strategy development – only the “Strategy Entry Logic” is used 
which creates the entry points (buy/short) – the exit logic is not yet used.
BTWFMgr analyzes for you how the market
responded – on average – to each series of entries 
generated by each input parameter permutation. This response is shown in the “Strategy
Potential Graph” – allowing you to see how far your strategy potential
moved and how long the move lasted.
BTWFMgr comes with an potential sample
“out of the box”, so you instantly can test drive BTWFMgr:
Start/Programs/Diamond Backtesting
…(BTWFMgr)/ Sample_Strategy_Potential_Analysis
Below are the main Treeview branches
showing the results (more details in further chapters):
Title 

Shows the details of the backtesting
data: 
Exit Optimization Results 

Showing the top 10 exit optimization
results 
Exit Parameter Analysis 

This section shows you the effect of
each exit logic element on the overall result. 
Strategy Potential Analysis 

This section allows you to find the
best strategy potential and also the extend and duration of the market
response on average(potential). The strategy potential for each
permutation in the backtest is shown and 
Strategy Input Parameter
Analysis 

This section allows you to track the
effect of each strategy input parameter on the potential and detect the
optimal parameter range. 
Strategy Context Variables Analysis 

Here you can detect if certain ranges
of additional context variables can filter out adverse results and improve
the performance, Example: 
To turn of the data collection OFF, simply
set nBTWFMgrExport to 0 in the strategy before running the backtest
optimization. This might be useful when for example the BTWFMgr trial period
has expired.
By
default BTWFMgr is installed to the C: drive into the C:/BTWFMgr folder.
All sub folders are created underneath this BTWFMgr “home” directory.
Each strategy has its own sub folder, with another subfolder for each symbol:
Example: C:\BTWFMgr\PSS_RSISample\@ES
Each backtest generates its own data files:
Examples:
C:\BTWFMgr\PSS_RSISample\@ES\015Min_20080912_132625.btwf1+2 (Equity data)
C:\BTWFMgr\PSS_RSISample\@ES\P_015Min_20081110_135126.btwf1+2 (Potential data)
{Result} is used below for the basic folder and
name of a backtest result  Example:
C:\BTWFMgr\PSS_RSISample\@ES\P_015Min_20081110_135126
Each
backtest generates the following files:
a)
BTWFMgr
Backtesting Definition File (btwf1)
contains general backtest information
b)
BTWFMgr
Backtesting Event Data File (btwf2)
contains the raw trading events, like buy/sell
c)
BTWFMgr
Backtesting Position Data File (btwf3)
contains the compacted trading information derived from the btwf2 file
d)
BTWFMgr
top results File ({Result}Best.CSV)
contains the result overview as a small spreadsheet,
which is used by the BTWFMgr overview module
to compare the various markets and timeframe results
In addition the
following files are generated by various functions:
·
BTWFMgr
Result Distribution File ({Result}Equ.CSV)
Listing all permutations for the selected sort criteria
·
BTWFMgr
Permutation Database ({Result}.MDB)
Exporting the statistics for each permutation to a Microsoft access database
for further processing
and even advanced SQL queries on your results
·
BTWFMgr
“long only” Position Data File ({Result}L.btwf3)
Extracted all the long positions from the {Result},
to identify the best results for the long side
·
BTWFMgr
“short only” Position Data File ({Result}S.btwf3)
Extracted all the short positions from the {Result}m
to identify the best results for the short
side
The Walk Forward
Analysis(WFA) generates its own folder and files within/underneath it:
·
BTWFMgr
Backtesting WFA Data File (btwf4)
contains the details of the Walk Forward Analysis(WFA)
Example: {Result}\P[Daily]+F[PFCheck3]+S[SortMix1].btwf4
·
BTWFMgr
Backtesting WFA equity overview File (CSV)
contains the WFA results sorted by ending OSA equity:
Example: {Result}\P[Daily]+F[PFCheck3]+S[SortMix1].csv
·
BTWFMgr
Backtesting WFA Result overview File (Results.CSV)
contains the details for each WFA result:
Example: {Result}\P[Daily]+F[PFCheck3]+S[SortMix1]Results.csv
·
BTWFMgr
WFA log files (CSV)
contains the various log file for each Walk Forward Analysis(WFA).
A new subfolder is generated:
Example: {Result}\P[Daily]+F[PFCheck3]+S[SortMix1]\XXX
A
special process allows you to migrate the BTWFMgr “home” directory to another
drive
(see
the “General/DRIVE” Configuration setting).
BTWFMgr allows you to easily change
almost all of its internal parameters,
so you can tailor the behavior to your needs.
To start the configuration, simply click on the icon
in the toolbar –or Ctrl+C –or
select or from the in the Menu: File/Backtesting Configuration and Preferences
(Ctrl+C)
All parameters are organized by topics:
·
TreeView
·
Smart Ranking Analysis
·
Best Trade Permutation Analysis
·
Walk Forward Optimization
·
Strategy Potential & Probability
Analysis
·
Initial Data Conversion
·
Position Exit Manager (PEM)
·
General
·
Chart
Below each topic the associated
parameters are attached. You can expand and collapse each topic,
by clicking on the plus or minus mark ( and ) in front of the topic.
To see a more detailed description 
click on the parameter,
and the right window section will show the description and current setting 
You can now:
·
Change the parameter value (simply
enter/select the new value and click on “Apply”)
·
Restore to the previous setting – after
an erroneous change for example – click on “Restore”
·
Switch to the parameter default value –
click on “Default”
Close the Configuration Window via the
“Close” button, the ESC key or the window button.
The new parameter settings is shown
with a star and activated next time you open BTWFMgr again.
A detailed description of each parameter is below in the “reference section”.
Below is a sample screen of the BTWFMgr Configuration and Preference Window:
(You can also edit the configuration parameters directly in the
C:/BTWFMgr/BTWFMgr.ini file)
BTWFMgr
offers many functions directly from the toolbar:
Below is a list of each function available from the toolbar:
ICON 
Shortcut 
Description 

Ctrl+O

Open
a new BTWFMgr data file (btwf1) 

File/Print 
Print
current view 

Ctrl+3 
Shows
Results in 3D View 

Ctrl+R 
Runs
the Walk Forward Analysis(WFA) in Equity Mode 

Function/Show
Data Properties 
Shows
the properties for the current data set 

Function/Split
into Long/Short 
Creates
new data sets for longs/shorts only 

Function/Recalculate
data set 
Recalculates
the current data set with changed parameters 

Function/Result
Distribution 
Activates
the Result Distribution function 

Function/Create
Overview 
Activates
the BTWFMgr overview module to 

Function/Prepare
your strategy 
Activates
the module to prepare your strategy for BTWFMgr 

Function/Configuration
and Preferences 
Set
Configuration and Preference parameter 


Switch
between Equity and Draw Down Chart Style 


Switch
Bollinger Band Display on/off 


Change
the Y Scale display to the next available style 
Help 
Activate
the BTWFMgr Help Module + License Manager 
BTWFMgr
comes with a free 15 days trial period – offering full functionality.
Check your BTWFMgr license status via: Help/BTWFMgr License Manager –or
Start/Programs/Diamond Backtesting …/License Manager
After the trail period expires  order your BTWFMgr subscription online at:
http://www.profsoftware.com/bt/order.htm
BTWFMgr
is available at a minimal cost in convenient 3, 6, 12 and 24 month
subscriptions or unlimited.
We offer discounts for longer subscriptions and grouping orders together for
other users or computers.
Over the long run the most economical is the outright/unlimited mode.
Each BTWFMgr license is on a per computer basis and not transferable.
You can request a trial extension – send us via email us the personal code:
Example: bw:c9hg0o+1djc8vd+pss@pobox.com
During the trial period  you have 10 free data conversions available.
Before the next of the 10 conversions is used up  a confirm box will appear:
click YES to use the next of the 10
trail data conversions  otherwise click NO.
Machine Replacement or Upgrade
While
BTWFMgr registration keys are purchased specifically for a single computer, PSS
may, at our discretion, allow you to transfer a serial number to a new or
upgraded computer. Such transfers will not be allowed at a rate of more than
once per 6month period, and proof of ownership of a new or upgraded computer
(such as an invoice or receipt) will be required along with a signed statement
requesting the transfer. Contact us at pss@pobox.com if you would like to
request a BTWFMgr license. This transfer option does not in any way alter or
override the End User License Agreement (EULA) stating that a registration key
is purchased for one machine only.
Below
we are presenting to you briefly the key concepts used in the BTWFMgr.
In few basic steps we show you how to quickly develop and test your strategy
with BTWFMgr.
Each Trading Strategy has two major elements:
·
Strategy Entry Logic – when to enter a position (buy/short)
·
Strategy Exit Logic – when to exit/reverse a position (sell/cover)
Strategy development starts in most cases with defining just the entry
condition(s) and then
translating those rules into the ”Strategy Entry Logic“ (i.e.
EasyLanguage® from TradeStation®).
BTWFMgr
allows you to optimize initially only the entry logic, without any exit logic
yet,
so you can identify initially the best entry points which yield the best “Strategy
Potential” (see below).
Once
the best “Strategy Potential” has been identified you can then move on
to the other “wing”
of
your strategy, which defines the most profitable way to exit the entries: the ”Strategy
Exit Logic“.
Only
when both “wings” of your strategy – entry AND exit  are independently strong
and healthy,
can your strategy “take flight” and can give you consistent profitable results.
BTWFMgr,
unlike all other backtesting software tools, does not force you to prematurely
include the exit logic, but allows you to first test only the entry logic
(Potential Mode) and then later
the entry AND exit (Equity Mode), so you can optimize each element
individually.
BTWFMgr
is collecting its backtesting data in the background while the Backtesting Optimization
is processing the data (designed to work currently with the TradeStation®)
8.X platform).
When the backtest process completes, BTWFMgr is automatically activated, and
will immediately convert and analyze and save the collected data, and then
present to you the results in several interactive graph,
so you can quickly identify the best results and apply the various functions to
analyze the results.
In
the quick strategy preparation – we will add a small section at the end of your
strategy code,
which will perform the data collection in the background for you. BTWFMgr can
do this one time modification of your strategy automatically, so you do not
have to be an EasyLanguage expert!
(More
details below in the “Strategy Preparation” chapter)
The
only reason for your strategy to enter a new position (buy/short) at the
various entry points, is,
that you expect – on average – that the market will move in the anticipated
direction, allowing you to realize a profit. This assumption can now be
precisely tested and verified using BTWFMgr’s
powerful and innovative approach – detecting the best Trading
Strategy Potential.
Lets look at one example:
In this example the market moves initially in the
anticipated direction (green line) and reaches after 60 bars and average gain
of $60 (on each trade). Then the market moves – on average  against you.
The “Strategy Potential Graph” shows you clearly:
BTWFMgr is calculating – for each strategy input permutation – the most best and worst strategy potential excursion and then presenting you a list of the best results.
Initially the optimal “Strategy Exit Logic” method is
not yet known, only the “Strategy Entry Logic”.
After we have identified the best strategy profit potentials, we can now detect
the best exit methods.
BTWFMgr come with an integrated “Position Exit Manager
(PEM)” which can evaluate 100,000 different exit method variations per
second, once all variations are tested BTWFMgr will then show you a list with
the best results. Then you can add the corresponding ”Strategy Exit Logic“
to your strategy.
In Potential Mode click on the OPT toolbar button and then initially create a
default Exit Definition:
Click on “Create Exit Definition”. You can then “Modify Exit Definition” or
directly “Run Exit Optimization”.
Now
with both the exit and entry logic  you are ready to rerun the TradeStation®
Optimization.
Set the nBTWFMGrExport strategy input parameter to 2 (Strategy Equity mode).
BTWFMgr will show you the best results
in several different lists:
·
Sorted by Max Equity
·
Sorted by Drawdown(%)
·
Sorted by PSSVolatility(%)
·
Sorted by Probability(%)
·
Sorted by ProfitFactor
·
Sorted by Smart Ranking1 – a custom
hybrid of above elements
·
Sorted by Smart Ranking2 – a custom
hybrid of above elements
·
Sorted by Smart Ranking3 – a custom
hybrid of above elements
BTWFMgr also shows you the optimal
ranges for each strategy input parameter and even
individual results for each specific strategy input parameter value!
In
this final step – to verify also using the powerful Walk Forward backtesting
method,
that your strategy has a realistic profit potential.
In many cases the curve fitted (fantastic) results in step4 are not likely to
repeat,
Which
leads usually to costly and discouraging trading results.
The new “Walk Forward backtesting method” can give you a much more
realistic preview and test of the true results your strategy can produce.
BTWFMgr
allows you to:
·
Use
any filter formula you devise
·
Use
any sort formula you want to test
·
Sort
all walk forward results identifying the best filter and sort method
·
Use
any insample and outofsample period setup
·
See
the outofsample results in one comprehensive equity graph or spreadsheet or
even 3D View
·
Show
the last period input parameter – which are then used in the actual live
trading
See
more information in a separate chapter “Walk Forward Analysis(WFA)” below.
BTWFMgr
contains also the advanced Walk Forward Analysis (WFA), which allows you
essentially to “firetest” your strategy performance, using historical data,
and to find the most robust and predictive approach and parameters. Walk
Forward Analysis (WFA) is available in BTWFMgr Equity mode.
Walk
Forward Analysis (WFA) is a powerful automated process – saving you hours or
days of manual errorprone calculations. The following WFA steps are performed:
·
Identify
the customizable “InSample” period (blue section below)
·
Extract
all candidates from the “InSample” period using the WFA Filter and
eliminate all undesired results. In some case ALL candidates are eliminated.
·
Find
the “best” “InSample” permutation based on your WFA Sort specifications
·
Shift
the focus to the next customizable “Outofsample” period (green sections
below)
which follows directly the “InSample” period
·
Apply
this “best” permutation to the “Outofsample” period,
thereby “firetesting” and checking if the “InSample” performance actually
continued.
·
Append
this “Outofsample” result to the overall WFA result.
·
Move
to the next “InSample” period until all data has been processed.
·
Show
the “best” strategy parameter for the future period (next “Outofsample”
period) (red below)
This WFA approach avoids the usual costly “curvefitting” phenomenon, when a
stellar performance, does in many case NOT continue in the future live trading!
Below is an example using a 4 week “InSample” period and one week
“Outofsample” period
with 7 walk forward steps:
Below
is a BTWFMgr example of a 4 week “InSample” period and one week
“Outofsample” period:
(WFA Optimization#3 of the WalkFwd.btwf1 sample data set)
Column 
Example 
Description 
Period# 
1 
Walk
Forward period, starting at 1 
ISAFrom 
20050606 
“InSample”
starting date (Jun 6^{th}, 2005) 
ISATo 
20050703 
“InSample”
ending date (Jul 3^{rd}, 2005) 
PermID 
2831 
“Best”
permutation ID for this ISA period 
ISAPL 
$1555.00 
Profit/Loss
within the “InSample” period (using PermID 2831) 
ISAHits 
24 
Number
of positions the “InSample” period 
ISAPF 
1.8 
ProfitFactor
within the “InSample” period 



OSAFrom 
20050704 
“OutofSample”
starting date (Jul 4^{th}, 2005) 
OSATo 
20050710 
“OutofSample”
ending date (Jul 10^{th}, 2005) 
OSAPL 
$135.00 
“OutofSample”
profit/loss within current period 
Equity 
$135.00 
Overall
OSA equity total 
OSAHits 
3 
Number
of positions the “OutofSample” period 
OSAPF 
1.6 
ProfitFactor
within the “OutofSample” period 



RSILength 
19 
Strategy
Parameters representing the ”best” Permutation ID 2831 
After the Walk Forward Analysis (WFA) you can then use the WFA Cluster Analysis
and 3D View to detect the best WFA trading parameters (more details below):
First
load the Equity result you would like to work with or click File/Open
WalkForward Sample File.
To activate the WFA Module:
·
click
on the (Optimize)
icon in the toolbar –or
·
Ctrl+R
–or
·
select
from the menu: Functions/Run Optimization (Ctrl+R)
Follow
the simple steps below to run your Walk Forward Analysis(WFA) (explained in more details below):
·
Select/Define
your WFA Filter
·
Select/Define
your WFA Sort
·
Select/Define
your WFA Period Setup
·
Select
any “Additional Diagnostic Options” – usually all are unchecked for faster
performance
·
Click
on the button and confirm
·
Wait
for the WFA to complete – BTWFMgr shows you the overall and individual progress
in %
·
While
WFA is running you can click on the button to pause or
abort.
to continue the WFA press the button
·
When
WFA has completed BTWFMgr will prompt you and
display the new WFA result branch in the treeview.
Below you can see the Walk Forward Setup module:
The
first step in the Walk forward Analysis(WFA), is to eliminate unwanted candidates
from the complete list of all trade permutations within each “InSample” range.
This process is repeated for each WFA period.
Select the Filter to be used in the WFA from the list of Filters in the Walk
Forward Setup module:
With the “Filter
Manager” you can create and manage your own filter(s).
You
can also select “NoFilter” from the list – if you do NOT want to filter any
permutations.
BTWFMgr
contains the easy “Filter Manager” interface– assisting you in creating/modifying
your WFA filters.
Simply click on the button in the Walk Forward
Setup module to activate the “Filter Manager:
BTWFMgr
will show the number of filter variations next to the selected Filter:
”24 Filter variations in 4 variables”
which is the product of: 2xPFMin, 3xPFMax,2x MaxGainCntMin, 2x MaxLossCntMin.
BTWFMgr
allows you to specify a formula, a simple logical expression, with a TRUE/FALSE
result.
TRUE=keep the permutation as a candidate, FALSE=discard the permutation in this
“InSample” range.
Below is a simple formula, which tests if the ProfitFactor ($PF) is at or above
1.0 (breakeven point):
($PF >= 1.0)
In
other words, we discard any permutations with a ProfitFactor below 1.0 (total
loss > total gains).
BTWFMgr
allows you even to go one step further and introduce “filter variables”, so you
can test several different filter variable values and determine which value(s)
yield the best overall results.
In the example below we replace the fixed “1.0” value with a new “PFMin”
variable (case sensitive):
($PF >= PFMin)
BTWFMgr allows you to define the values for “PFMin”:
PFMin=0.9 to 1.3 step 0.1
BTWFMgr
will run 5 tests with the following “PFMin” values:
0.9, 1.0, 1.1, 1.2, 1.3 (0.9 and 1.3 in 0.1 steps)
To see a list of available “Filter Statistics Variables” click on
or select from the list below:
Below
is a table of all system variables available for the Filter and Sort Formula:
$Equ 
Ending Equity 
$CntGain 
Number of Winning Trades 
$CntLoss 
Number of Losing Trades 
$CntTrd 
Number of all Trades 
$GrossGain 
Total of all Gains 
$GrossLoss 
Total of all Losses 
$AvgGainAmt 
Average Winner Amount 
$AvgLossAmt 
Average Loser Amount 
$MaxGainAmt 
Max Winner Amount 
$MaxLossAmt 
Max Loser Amount 
$MaxGainCnt 
Max Winner in a row 
$MaxLossCnt 
Max Losers in a row 
$PF 
ProfitFactor ($GrossGain / $GrossLoss) 
$PFAvg 
ProfitFactor ($AvgGainAmt / $AvgLossAmt) 
$Prob 
Probability(%)$CntGain/$CntTrd 
$MaxDDAmt 
Max Drawdown Amount (negative) 
$MaxDDPer 
Max Drawdown (% of Starting Capital) 
$DDAvg 
Drawdown average (% of ending Equiy) 
$StdDevAmt 
Standard Deviation Amount of all Trades
Profit/Losses Amounts 
$StdDevPer 
Standard Deviation (% of Starting
Capital) 
$VolatilityAmt 
Volatility
Amount = Avg distance$ from straight Income line 
$VolatilityNet 
Volatility
(% of Starting Capital) 


$RankEqu 
Equity Rank (100%=best/highest,
0%=worst/lowest) 
$RankMaxDD 
Drawdown Rank (100%=best/smallest,
0%=lowest/highest) 
$RankVolatility 
Volatility Rank (100%=best/smallest,
0%=lowest/highest) 
$RankProb 
Probability Rank (100%=best/highest,
0%=worst/lowest) 
$RankPFGross 
ProfitFactor ($PF) Rank
(100%=best/highest, 0%=worst/lowest) 
$RankCombo1 
Smart Ranking Result1(%)
(100%=best/highest, 0%=worst/lowest) 
$RankCombo2 
Smart Ranking Result2(%)
(100%=best/highest, 0%=worst/lowest) 
$RankCombo3 
Smart Ranking Result3(%)
(100%=best/highest, 0%=worst/lowest) 
To
check the ranges and see actual values for each of these system variables,
right click on any permutation
in the treeview and select “Export Statistics to Spreadsheet (All
Permutations)“.
BTWFMgr
offers even the use of math functions in your WFA Filter formula, here is the
“PROM: filter formula:
(($AvgGainAmt*($CntGainsqrt($CntGain)))/($AvgLossAmt*($CntLosssqrt($CntLoss))))>
PROMMin
Below is a table of available math function in the WFA Filter Formula:
abs(x) 
Absolute Value: abs(5) = 5 
foor(x) 
largest integer that is less than or
equal to x: floor(2.8)=2, floor(2.8)=3 
sqrt(x) 
Square root(x) 
sign(x) 
Sign: sign(3)=1, sign(0)=0, sign(6)=+1 
exp(x) 
Exponential: exp( 2.302585 ) = 10.000000 
pow(x,y) 
Power of: pow(2,3) =8.00000000 
ln(x) 
Natural
logarithms: ln(9000) = 9.104980 
log(x) 
Decimal
logarithms: log(9000) = 3.954243 
log2(x) 
Binary logarithms : log2(4) = 2.000 
sin(x) 
Sinus:
sin(1.570796) = 1.000000 
cos(x) 
Cosinus: cos(1.570796) = 0.000000 
cmple(x,y) 
Compare if less or equal: cmple(2,3)=1, cmple(2,2)=1, cmple(2,1)=0 
cmplt(x,y) 
Compare if less: cmplt(2,3)=1, cmplt(2,2)=0,
cmplt(2,1)=0 
cmpge(x,y) 
Compare if greater or equal:
cmpge(2,3)=0, cmple(2,2)=1, cmple(2,1)=1 
cmpgt(x,y) 
Compare if greater: cmpgt(2,3)=0,
cmpgt(2,2)=0, cmple(2,1)=1 
To
create a new filter click on “Create a New Filter”:
·
Choose
if you want to clone the current filter or
create a blank new filter
·
Enter
the new formula
·
Click
on to see available
statistics variables.
·
If
you are using variables  click on “Add Variable”
·
Enter
the new variable name
·
A
new variable with a single “0” value will be added
·
Click
on the “0” value in the “Test Range” column
·
BTWFMgr
will switch to edit cell mode
·
Enter
the new list of values, you have two formats available:
1.0 to 2.0 step 0.2 or
1, 1.3, 1.5, 2
·
When
you have added all variables and the formula click on “VERIFY”
·
Enter
any comment or description for the new filter
·
Correct
any errors
·
Finally
click on “SAVE” to actually save the new filter to the file
·
From
now on the new Filter will appear in the pull down menu.
·
If
you want to view the actual filter file content – click on “Edit Filter
directly”
Select
the filter to be deleted and click on “Delete Filter” – confirm the deleting.
BTWFMgr
stores each WFA filter in a separate text file.
Here is an example of the “PFCheck:” filter file – located at
C:/BTWFMgr/PFCheck.btwff”:
Formula=($PF >= PFMin)
AND ($PF <= PFMax) AND ($MaxGainCnt > MaxGainCntMin) AND ($MaxLossCnt
< MaxLossCntMax)
PFMin=1.0 to 1.2
step 0.2
PFMax=1.8 to 2.2
step 0.2
MaxGainCntMin=6,5
MaxLossCntMax=6,5
;====== SAMPLE
FILTER FILE ==========================
;====== THIS
WILL FILTER BASED ON:
; ProfitFactor
($PF) and
; Consequtive
Win/lose runs ($MaxGainCnt/$MaxLossCnt)
The first line is always the Formula
Formula=($PF
>= PFMin) AND ($PF <= PFMax) AND ($MaxGainCnt > MaxGainCntMin) AND
($MaxLossCnt < MaxLossCntMax)
Then follow all the Filter Variables with their ranges
PFMin=1.0 to 1.2 step 0.2
PFMax=1.8 to 2.2
step 0.2
MaxGainCntMin=6,5
MaxLossCntMax=6,5
At the end is your description/comment – always starting with a semicolon:
;====== SAMPLE FILTER FILE
==========================
;====== THIS
WILL FILTER BASED ON:
; ProfitFactor
($PF) and
; Consequtive
Win/lose runs ($MaxGainCnt/$MaxLossCnt)
If
you are confident enough you could directly create and edit these filter file
(C:/BTWFMgr/*.btwff)
After
the WFA has selected all viable candidates and discarded the rest in the Filter
Step1,
We
still have usually many candidates left.
In this WFA sorting step, BTWFMgr will rearrange the candidates in a
customizable sort order.
The permutation/candidate showing up at the TOP is the ONE winner,
which is then used in for the “Outof –sample” calculation.
Each Sort Definition can contain one or several sort methods.
Below is an example of 4 different sorts:
Sort1=$MaxTrdLoss/5 DESC,$Equity,$MaxDDAmt
Sort2=$Equity,$MaxDDAmt,$MaxTrdLoss DESC
Sort3=$PF,$Equity,$MaxTrdLoss DESC
Sort4=$PFAvg,$Equity,$MaxTrdLoss DESC
Sort4=$PFAvg,$Equity,$MaxTrdLoss
DESC
Each WFASoft Definition is defined in a text file (Example
C:/BTWFMgr/SoftMix1.btwfs).
The “Walk Forward Sort Manager” is assisting you in managing and defining your
sort criterias:
BTWFMgr
allows you several different ways to design the various insample(ISA) and
outof sample(OSA) testing periods. BTWFMgr comes with many predefined period
setups,
but you can define and add any custom period setup!
To view the actual WFA period – click on (see examples below)
You can even specify your own custom shift (in days) in the configuration:
Walk Forward Optimization/WFODayShift=0
a) Number of Walk Forward Runs and OSA Percent
You specify the number of Runs and the percent of outofsample days
Example: 10 Runs with 10% outofsample
b) Number of ISA/OSA Days
Example: 100 days ISA periods with 20 day outofsample periods
c)
Number of ISA/OSA Weeks
ISA periods always start on a Monday.
Example: 8 weeks ISA periods with 2 weeks outofsample periods
BTWFMgr
allows you even to select a sequence of period setups,
so you can test which period setup performs the best for your strategies!
You can either select one of the predefined Period Sequences (Daily, Weekly
etc) –or
design your own – press the “Seq Mgr” button – and the ”Walk Forward Period
Sequence Manager” will open!
The following functions are available:
a) Single Add: click one period on left – then click the Add button
b) Group Add: click
your selection on the left – then click the Add button
c)
All Add: click on the Add All button
d) Generate custom Sequence: enter the “OutofSample(%)” and “Nbr of Runs”
list;
then click the “Generate a Walk Forward Sequence”
e)
Single Delete: click one period on right– then click the Delete button
f) All Delete: click
on the Delete All button
Remember to save any
newly created Period selection – press the Save putton:
 then enter the new Period
Sequence Name.
Usually
all check boxes in the “Additional Diagnostic Options” section are left
unchecked
to achieved maximum speed for the Walk Forward Analysis:
In some cases you might want to review and track internal WFA calculations and
enable some diagnostic options, which are also saved to the new WFA sub folder.
To
finally start the Walk Forward Analysis (WFA) – simply press the green “Start
Run” button:
And
confirm the new Walk Forward Analysis Run:
You can now abort or pause the WFA run by pressing the button.
A
box appears were you can choose:
As
BTWFMgr calculates the many WFA permutations  the progress is shown to you in
the cyan bars
For
each level:
·
Filter
(current filter processed with in the current Date Range)
·
Range
(current Date Range within the current period)
Each period definition creates several date ranges for the
InSample/OutofSample periods.
Click on “View Periods” to see the date ranges being created for the WFA
·
Period
(current Period as defined in the Period Sequence, unless a single period has
been selected)
·
Overall
progress at the button
Example
of running WFA:
When
the WFA has completed, BTWFMgr automatically add a new tree branch, with blue
squares, at the top.
The
WFA results are shown first sorted by 6 different criteria (Equity, Drawdown,
Volatility, Probability, ProfitFactor and Robustness). Then follow the Filter
and Sort Variations used in this WFA and the In/OutSample periods calculated –
Below is an example:
Walk Forward Analysis – Result Viewing
BTWFMgr
offers many ways to view the WFA results and identify the best WFA logic:
BTWFMgr
is showing by default the top 10 WFA results, just click on any of the results
to view the equity.
The remaining results are shown in the “XXX Other Optimizations” branch.
If you want to see more than 10 top results – changed the configuration
parameter “MaxWFOOpt”.
Click
on the (+) in front of the “Sorted by Probability” branch and BTWFMgr will
expand the branch
and show the list of the top 10 results:
If
you want to see more than 10 top results – changed the configuration parameter
“MaxWFOOpt”
Click
on the (+) in front of the “Sorted by XXX” branch and BTWFMgr will expand the
branch
and show the list of the top 10 results – similar to Equity and Probability.
Right
click on any WFA results and you will see the WFA popup menu – then click on
”Walk Forward ResultCluster Analysis” (or press Ctrl+A) or from the Menu
Function/Cluster Analysis:
The Walk Forward Result+Cluster analysis box will appear:
Select
the criteria you would like to view (i.e. Net PL$) and click on “3D View”.
Now
you can choose if you want to plot the Filter and Sort variations –OR In/OutSample
variations:
The
“Filter/Sort Matrix” will use the Filter and Sort variations for the X and Z axis.
The
“Period Matrix” will use the In/OutSample variations for the X and Z axis.
The
Y Axis is defined by your selected criteria abive.
Below
is an 3D WFA Result example, showing:
·
X
Axis: 4 sort variations plus the 5^{th} Sort Average results
·
Z
Axis: 24 filter variations plus the 25^{th} Sort Average results
·
Y
Axis: The Net Profit+Loss($) (or whichever criteria you selected)
·
To
Zoom in – rotate the mouse wheel away from you
·
To
Zoom out– rotate the mouse wheel towards you
·
To
Rotate to the left – click and hold left mouse button and move slowly to the
right
·
To
Rotate to the right– click and hold left mouse button and move slowly to the
left
·
To
Rotate to the top– click and hold left mouse button and move slowly down
·
To
Rotate to the bottom– click and hold left mouse button and move slowly up
·
To
move the center – click on hold the right mouse button and move in the desired
direction
·
To
capture/print the view – click on the print button in the toolbar
Here
we give you a more detailed look at the actual steps the WalkForwardAnalysis
(WFA) process.
WFA will first create a new folder were all results are saved with the input
filename:
Example: Results for Input file:
“C:\BTWFMgr\PSS_RSISample\@ES\015Min_20080829_123503.btwf1”
are all saved to folder “C:\BTWFMgr\PSS_RSISample\@ES\015Min_20080829_123503”
The general WFAVerbose level determines the processing diagnostic details
level;
Which
all off for highest WFA Speed!
The
following abbreviations are used:
WFA=Walk Forward Analysis, ISA=In Sample Range, OSA=Out of Sample Range
The
following steps occur when for each WFA process:
·
Main1:
Load the period sequence definitions (from selected *.btwfl file, i.e.
Daily.btwfl)
·
Main2:
Check if for each period sequence definition enough data is present
·
Main3:
Create a matrix for each filter/sort and period variation
o Period1: Load next period
sequence definition (i.e. 10 Runs OSA 5%)
o Period2: Create the
In/OutSample date ranges – as shown in “View Periods” function
(i.e. create 10 date ranges for current data an OSA portion of 5%)
o Period3: Process the
first/next period definition
§ Range1: Process first/net
date range  create the subfolder for results:
Example: P001[30D_10D]+F[PFCheckTest]+S[SortMix1] = {PeriodFolder}
§ Calculate the statistics for
each Trade Permutation in the current ISA date range
§ Range2: Calculate the
relative ranking values
§ Range3: If requested
(SaveTrdPerm switch) export the statistics of each Trade Permutation to
the range result file in a sub folder for this period def, Example:
{PeriodFolder}/TradePermutations/ Rng0001_20080109_20080207.csv
§ Range4: Calculate the
specified filter condition to pass/fail each trade permutation
§ Range5: Sort the remaining
filtered candidates (unless ALL permutations filtered out)
§ Range6: If requested
(SaveOptPerm switch) exports the
{PeriodFolder}/Hits/OPT0000001_P001_F0001_S0001_20080109.csv
§ Range6: Establish the final
trade permutation ID
§ Range7: If requested
(SaveRunLog switch) exports the best result to:
{PeriodFolder}/Rng0001_20080109_20080207.csv
§ Repeat Range1Range7 for all
remaining date ranges for this period
o Period4: Accumulate the OSA
Results and calculate the OSA statistics
o Period5: If requested
(SaveRunLog switch) exports the best result to:
{PeriodFolder}/ OPT0000008_P001_F0002_S0004.csv
o Repeat Period1 Period5 for
all remaining Period Definitions
·
Main4:
Save all WFA calculations to a binary file (*.btwf4)
·
Main5:
Post Process overall WFA Results
·
ll results are saved in a newly created folder – using the filename – Example:
WFA Results for: C:\BTWFMgr\PSS_RSISample\@ES\015Min_20080829_123503.btwf1
will be saved in new folder: “C:\BTWFMgr\PSS_RSISample\@ES\015Min_20080829_123503”
Within that folder the following sub folder are generated:
TradePermutations\P001_XXX – Results for Period Sequence #1
TradePermutations\P001_XXX\R0001_20080109_20080207_0000084Permutations.csv 
R0001=Range#1, 20080109=ISA Start Date, 20080207=ISA End Date
P[30D_10D]+F[PFCheckTest]+S[SortMix1] – WFA Results for:
Period “30D_10D” Filter “PFCheckTest” and Sort “SortMix1”
”SaveOptPerm” switch:
P[30D_10D]+F[PFCheckTest]+S[SortMix1]\Hits – WFA Filter Results
P[30D_10D]+F[PFCheckTest]+S[SortMix1]\Hits\SEQ0000001_P001_F0001_S0001_20080208.csv
SEQ0000001=Overall Sequence, P001=Period#1, F0001=Filter#1, S0001=Sort#1
20080208=ISA Start
”SaveRunLog” switch:
SaveRunLog
In
this section we illustrate in more detail how to use the various powerful
BTWFMgr functions.
Some functions might only apply only to Equity Mode (Equ) and/or Potential Mode
(Pot)
After
you run an optimization, the results contains usually long and short trades in
the many permutations. BTWFMgr will detect the overall best Equity, Drawdown
etc using long and short
and show them in the “Sort by XXX” green square branches:
In many cases you might need to split the long and short results into separate
files,
Thereby creating their own
Backtesting context with new “Sorted by XXX” lists!
You
can do the L/S Split
by simply clicking on the LS
icon in the toolbar:
Next BTWFMgr will confirm
click
YES to continue:
Next BTWFMgr will proceed to generate
two new BTWFMgr result files,
And when finished, give you the option
to view the new “Long Only” (WalkFwdL) result:
To view the “Short only” result – click on the new C:\BTWFMgr\WalkFwdS.btwf1
file.
BTWFMgr
(as most other software tools) sort all results by the ENDING Equity,
Which
is considered the best performance.
But
as you might have noticed, many times the best performance, is not always
continuing that previous performance – below is an example how a stellar
performance:
can
turn into a big drawdown
With
BTWFMgr you can now to doublecheck and research this phenomenon using your own
results.
Just define an earlier reference point within the equity curve – which is used
to sort the results,
instead of the usual 100% ENDING equity point.
In effect you virtually “cut off” a portion of last trades – just for the
sorting, the data Is not really lost.
(This
“cutoff” function only applies to the Equity Mode)
Simply click on the Recaclulate Icon:
Then
enter the new point of reference (instead of the usual 100% at the end):
To
start the conversion click “Recalculate”.
The new equity curves will show a new magenta line; connecting the cutoff point
with the ending result:
Click
on any performance result and you can easily check visually –
if
the previous performance actually continued.
This is similar to the the “InSample” and “OutSample” segments used in the Walk
Forward Analysis(WFA),
If the Equity results are using an early “cutoff” point
BTWFMgr will show the cutoff percentage in the header:
In
some cases you might want to use a custom spreadsheet, to identify and analyze
the backtesting trade permutations. BTWFmgr allows you to export all the
relevant statistical values for each permutation (or just the current one) to a
spreadsheet.
Simply right click on the permutation and select “Export Statistics”:
BTWFMgr
will generate
a new spreadsheet file for the current file – with StatsALL appended 
and ask you if you would like to view the new spreadsheet file (CSV format):
Click YES to see the
new spreadsheet, were you can now apply your custom analysis/sort etc:
This Statistic Export can also assist you in designing custom WFA filter
formulas!
Instead of all permutations you can also select the single export –for just the
current permutation.
In order for BTWFMgr to analyze your strategy – BTWFMgr has to collect the
relevant data generated during your TradeStation®
Optimization runs.
To accomplish this BTWFMgr adds a small
section to the end of your EasyLanguage® strategy code.
Before BTWFMgr can automatically add the data collection section – open the strategy code:
·
Open TradeStation®:
Start/Programs/TradeStation 8.1 (Build XXX)
·
Close all workspaces
·
Open the strategy EasyLanguage Code
File/Open EasyLanguage Document (or Ctrl+E)
·
Select "Strategy" in the
Select Analysis type pull down list (at the top)
·
In the new list scroll down to your
Strategy and click the Strategy Name
then click on the Open button (as an example we use the PSS_T_Sample1 strategy
here …)
·
You might need to enter a protective
password to open the Strategy Code
·
A new Strategy EasyLanguage window has
opened:
Now we have
prepared TradeStation® for BTWFMgr to automatically modify your strategy.
·
Open BTWFMgr:
Start/Programs/Diamond Backtesting with Walk Forward Manager
(BTWFMgr)/Backtesting with Walk Forward Manager (BTWFMgr)
·
Once BTWFMgr has opened – click on the
cyan exclamation mark in
the tool bar:
(or select File/Strategy Preparation from the Main Menu)
·
A new window will open – showing your
strategy name – click on Prepare:
·
BTWFMgr is now presenting you with all
numeric strategy input parameter,
which can be included in the data collection during the TradeStation®
optimization:
(indicated by the X in the Optimization column)
Select ALL inputs you might use in any TradeStation® optimization (usually
all).
Just double click on the input row and BTWFMgr will select/deselect it.
·
You can change the sequence of an input
with the up and down button
·
Now click on the modify button to let
BTWFMgr add the new data collection section
·
Now BTWFMgr will automatically modify
the strategy code for you and
after a few seconds show the confirmation box – click on OK:
·
Now you are ready to run the regular
TradeStation® Optimization on your strategy,
which will now collect the relevant data for you, so you can then analyze it in
BTWFMgr.
The initial Strategy Potential mode (nBTWFMgrExport=1) is automatically
preselected for you.
Below is a sample of the BTWFMgr data collection section.
When you develop your strategy, you
usually want to research also the effect of additional context input values on
the overall results. With BTWFMgr you can now check if certain ranges or values
will improve the overall strategy results and by filtering out bad entries.
BTWFMgr adds for you automatically already the TimeOfDay context value.
Example:
Research of a certain range of the moving average angles might be connected to
better results.
Below are the steps to add an additional context variable:
·
Add Context Variable Name
Add one line for each new context variable after the line
WalkForwardVar = WalkForwardVar + "/*TimeOfDay";
Here is one example  adding an angle:
WalkForwardVar = WalkForwardVar + "/ExpAngle";
(the * indicates the first context variable name):
·
Increase Argument Counter
Increase the input counter by the number of newly added context values,
below the line:
arrPSSBTVal[0] = 6; // Number of tracked Strategy Inputs
if nBTWFMgrExport = 1 OR (nBTWFMgrExport = 3 AND CurrentShares = 1818) then
arrPSSBTVal[0] = 7; // add Number of
Context Variables
In our example: we increase by one  from 7 to 8
·
Add Context Variables to
Function Call
Add the actual context values to the BTWFMgr data capture function argument
list, after the line:
arrPSSBTInp[6] = Time; // Context Input#01
Here is one example  adding an angle:
arrPSSBTInp[7] = AnglePercent; // Context Input#02
·
Verify the EasyLanguage
After modifying he strategy code verify the EasyLanguage code (Tools/Verify)
·
ReRun TradeStation®
optimization
Now you can rerun the TradeStation® Optimization,
which will now collect also the data at each entry also the added context
variable values.
More information at: http://www.profsoftware.com/bt/eldmode.htm
BTWFMgr will show the result for each context variable in the "Context
Variables" section.
Below is a sample of the BTWFMgr data
collection section – with the two added context variables:
Time, // Context Input#01
AnglePercent, // Context Input#02
//====== BTWFMGR_F2D83D3A_BBDB_447E_B0FE_209ED95E4E3F
=============
//====== WALKFORMWARDOPTIMIZATION DATA COLLECTION
SECTION ======
Vars: nRetWFO(0),WalkForwardVar("");
if nBTWFMgrExport > 0 then begin
if GetAppInfo(aiOptimizing) = 1 then begin
if
WalkForwardVar = "" then begin
WalkForwardVar
= "RSILength";
WalkForwardVar = WalkForwardVar + "/OverSold";
WalkForwardVar = WalkForwardVar + "/OverBought";
WalkForwardVar = WalkForwardVar + "/EMALength";
WalkForwardVar = WalkForwardVar + "/LossAmt";
WalkForwardVar = WalkForwardVar +
"/GainAmt";
if
nBTWFMgrExport = 1 then begin
WalkForwardVar = WalkForwardVar + "/*TimeOfDay";
WalkForwardVar = WalkForwardVar + "/ExpAngle";
end;
end;
if
nBTWFMgrExport = 2 then begin
nRetWFO = PSS_F_WFO10("PSS_T_Sample1", //Strategy
WalkForwardVar,
// Names of Inputs
6,
// each selected Input Variable below
RSILength, // Strategy Input#01
OverSold,
// Strategy Input#02
OverBought,
// Strategy Input#03
EMALength,
// Strategy Input#04
LossAmt, // Strategy
Input#05
GainAmt
// Strategy Input#06
,0,0,0,0);
end;
if
nBTWFMgrExport = 1 then begin
nRetWFO = PSS_F_WFO10("PSS_T_Sample1", //Strategy
WalkForwardVar, // Names of Inputs
8,
// each selected Input Variable below
RSILength,
// Strategy Input#01
OverSold, // Strategy Input#02
OverBought,
// Strategy Input#03
EMALength,
// Strategy Input#04
LossAmt,
// Strategy Input#05
GainAmt, // Strategy
Input#06
Time,
// Context Input#01
AnglePercent, // Context Input#02
0,0);
end;
end;
If
nBTWFMgrExport = 1 then begin
If
marketposition = 1 then
Sell ("Lbt") next bar Market;
If
marketposition = 1 then
Buy to Cover ("Sbt") next bar Market;
end;
end;
//======
BTWFMGR_F2D83D3A_BBDB_447E_B0FE_209ED95E4E3F =============
After
you have prepared your strategy and the TradeStation Optimization has
completed,
BTWFMgr will automatically starts and perform automatically the initial data
conversion and then
present to you the Strategy Potential Analysis. (make sure you have
nBTWFMGrExport set to 1).
Analyzing
the Strategy Per Trade Potential
Analyzing
the Strategy Yearly Potential
Analyzing
the Strategy Parameter Potentials
Analyzing
the Strategy Context Variables
(under
construction …)
·
Exit Optimization Results
Showing the top 10 results  sorted by the best average return per trade:
·
The remaining results are collected behind the “Other Exit
Optimizations” item:
How many exit optimizations are shown is defined in
the “MaxExitResults” configuration parameter (2000) in the TreeView section.
·
Exit Parameter Analysis
Showing for each exit parameter the individual results
After
you have identified the best Strategy Potential for Long and Short trades,
you can now let the “Position Exit Manager (PEM)” calculate many
different exit scenarios,
so you can identify the best
“Strategy Exit Logic”.
Usually Longs and Shorts have different optimal exit parameters.
Since the integrated “Position Exit Manager (PEM)” is using fast object
oriented C++ code,
it can process 100,000 exit variations per second (~1000 time faster than the
regular backtesting)!
First click on the result with the best potential, which actualizes the
permutation ID.
Then Click on the button in the
toolbar, which brings up the Optimization box:
The current permutation ID (1018) is shown in:
which
determines all the positions to be tested.
Then
click on to create a new
default template tailored to the current potential.
This will create a new exit definition:
Each line defines a type of exit with a testing range
(see details about exit types below).
You can select various processing parameters:
Finally you start the processing – click on:
Which
will process the 180,224 exit permutations
for each of the 579 positions in Trade Perm ID 1018
In
only a few minutes all 88.4 million scenarios are completed:
After
the optimization is complete a new exit result branch will appear in the
treeview:
Analyzing
the Strategy Per Trade Potential
BTWFMgr
allows you to easily change and almost all of its internal parameters,
so you can tailor the behavior to your needs.
To start the configuration, simply click on the icon in the toolbar –or Ctrl+C
–or
select or from the in the Menu: File/Backtesting Configuration and Preferences
(Ctrl+C)
All parameters are organized by topics:
·
TreeView
·
Smart Ranking Analysis
·
Best Trade Permutation Analysis
·
Walk Forward Optimization
·
Strategy Potential & Probability
Analysis
·
Initial Data Conversion
·
Position Exit Manager (PEM)
·
General
·
Chart
Below
each topic the associated parameters are attached. You can expand and collapse
each topic,
by clicking on the plus or minus mark (
and )
in front of the topic.
To
see a more detailed description  click on the parameter,
and the right window section will show the description and current setting 
You can now:
·
Change the parameter value (simply
enter/select the new value and click on “Apply”)
·
Restore to the previous setting – after
an erroneous change for example – click on “Restore”
·
Switch to the parameter default value –
click on “Default”
Close
the Configuration Window via the “Close” button, the ESC key or the window button.
The
new parameter settings is shown with a star and activated next time you open
BTWFMgr again.
A detailed description of each parameter section is presented below.
Below is a sample screen of the BTWFMgr Configuration and Preference WindowBelow is a list of all the available BTWFMgr Configuration settings you
can adjust to your needs.
(You can also edit the configuration parameters directly in the C:/BTWFMgr/BTWFMgr.ini
file)
In this section you can
adjust settings related to the BTWFMgr treeview window:
Name 
Default 
Description 
AttachInputs 
YES 

AttachPermToSubInput 
NO 
Attach automatically all
relevant Trade Permutations to each Sub Input variation. You can of cause
still attach/detach the permutations manually, NO is recommended for large
Permutation sets (30,000+) 
AttachSubInputsMaxPerm 
10000 
Maximum number of Trade
permutations before the automatic attachment of Trade Permutations to inputs
is turned off 
AttachSeqTrdPerm 
YES 
Automatically attach ALL
Trade Permutations in Sequential order after the Best Results display 
AttachPosSeq 
NO 
Automatically attach the
positions to each Trade Permutation in the Sequential Permutation List.
Warning: YES can consume a lot of memory with large Permutation sets (30000+) 
MaxExitResults 
2000 
Maximum number of Position
Exit(PEM) Results shown in the TreeView or potential results 
EquTimeLength 
30 
Define the time length for
automatic Time of Day Analysis 
TreeWidth 
360 
Define the default width
in pixels of the treeview window (360=default) 
In this section you can
adjust settings related to the BTWFMgr Smart Ranking Analysis:
Name 
Default 
Description 
EquRankDef1 
3x$RankEqu, 
Define the Equity Smart
Ranking#1 Analysis formula 
EquRankDef2 
4x$RankProb 
Define the Equity Smart
Ranking#2 Analysis formula 
EquRankDef3 
2x$RankMaxDD 
Define the Equity Smart
Ranking#3 Analysis formula 
MinEquityPercent 
20.0 
Cutoff percentage in the
Equity Ranking for the Smart Ranking Candidates 
PFMin 
0.2 
Define the minimum
ProfitFactor(PF) used the filter out Trade Permutation results for the Smart
Ranking Analysis 
PFMax 
4.0 
Define the maximum
ProfitFactor(PF) used the filter out Trade Permutation results for the Smart
Ranking Analysis 
PotRankDef1 
2x$RankAvg 
Define the Potential Smart
Ranking#1 Analysis formula 
PotRankDef2 
2x$RankYrAvg 
Define the Potential Smart
Ranking#2 Analysis formula 
PotRankDef3 
1x$RankAvg 
Define the Potential Smart
Ranking#3 Analysis formula 
In this section you can
adjust settings related to the BTWFMgr Best Trade Permutation Analysis:
Name 
Default 
Description 
CalcBestTrdPerm 
YES 
Enable or disable Best
& Smart Trade Result display 
SkipLoosers 
NO 
Skip loosing Equity
Permutations (Default=include them) 
CacheBestTrdPerm 
YES 
Enable or disable Best
Trade Result caching to speedup the load 
AttachBestPos 
NO 
Automatically attach the
positions to each Trade Permutation in the Best Result display. Warning: YES
can consume a lot of memory with large Permutation sets (30000+) 
MaxPermBest 
20 
Number of Trade
Permutations shown in the Best Result List 
MaxPermBest2 
1000 
Total number of Trade
Permutations shown in the remaining Best Result section 
MaxPermBestExport 
100 
Total number of exported
best Trade Permutations to CSV Spreadsheet file for the BTWFMgr Overview
module 
MakeBestUnique 
YES 
Eliminate Duplicate
Results (YES) in the Best Trade Result display 
CompareMaxDiffMinutes 
30 
When Comparing Position
Lists define the\nmaximum Number of entry time differences (in Minutes) 
In this section you can
adjust settings related to the BTWFMgr Walk Forward Optimization and Analysis:
Name 
Default 
Description 
MaxWFOOpt 
10 
Maximum number of Walk
Forward Optimizations to show directly in the Treeview. The remaining results
are shown in the Other results branch 
OSAStyle 
0 
Define how Walk Forward
Out Of Sample Positions are handled which cross/overlap across the begin or
end of the OSA Period: 
OSAStyleVerbose 
YES 
Switch OSA Overlap Check
verbose mode ON/OFF 
SaveRunSet 
NO 
Save each Walk Forward
Period Result 
SaveTrdPerm 
NO 
Save the Trade
Permutations of each Walk Forward Period 
SaveOptPerm 
NO 
Save the each filtered
Trade Permutations of each Walk Forward Period 
SaveRunLog 
NO 
Total number of exported
best Trade Permutations to CSV Spreadsheet file for the BTWFMgr Overview
module 
WFODayShift 
0 
Number of Days to shift
from the calculated default dates: 
WFOAddPeriod 
1 
Number of periods to
append without historical data 
AUTOSTART 
1 
Enable the automatic
conversion of backtesting data 
AUTOEXIT 
0 
Enable the automatic exit
after conversions (set to 1) 
AUTOWFO 
0 
Enable the automatic
WalkForward Optimization with the last used WFA parameters after the data is
converted/loaded (set to 1) 
In this section you can
adjust settings related to the BTWFMgr Potential Analysis:
Name 
Default 
Description 
CommissionAndSlippage 
15.00$ 
Set the amount of the
commission (incl. slippage) per position to get more realistic results 
PotentialMaxLoss 
400$ 
Define the maximum loss
during potential scanning 
RRScope 
200 
Define the maximum scope
(number of bars) available for the Strategy Potential analysis 
RREOD 
0 
Define the End of Day Exit
Time for Intraday strategies to avoid holding overnight (0= no exit,
1615=4:15PM) 
RREOW 
0 
Define the End of Week
Exit Time for Intraday strategies to avoid holding over a Weekend 
RRNetPotential 
YES 
Use Net Best Potential
(Best minus worst potential up to that point) 
RRMinPos 
20 
Minimum number of
Positions required for a Trade Permutation in order to qualify for the Best
Trade Result display 
RRMinDistPos 
10 
If positions entries are
too close  skip during Strategy Potential calculations  minimum distance is
defined here 
MinRRCnt 
8 
Minimum number of Position
samples needed for Result display 
RRAvgPer 
10 
Number of bars used for
the Strategy Potential Average calculation 
RRPercentFactor 
100 
Factor to magnify the
Percent display in the Y Scale in the Strategy Potential Chart 
MaxContextVariations 
15 
Maximum number of
different Context Input Values for direct display of the averages for each
Input Value 
MaxVirtualVariations 
15 
Number of different
virtual Context Input Values 
ProbLevels 
50,100,150$ 
Probability Analysis
trigger levels as Amount in $ 
ProbLevel 
2 
Probability Level to be
used for global Probability Average (2=middle level = $100) 
ProbAvgPer 
5 
Number of bars used for
the Strategy Probability Average calculation 
ChkRRScope 
NO 
Check if current RRSCope
matches the selected refresh filelist 
In this section you can
adjust settings related to the one time initial dataconversion
from the btwf2(Trade events) to the final btwf3(Positions) data file:
Name 
Default 
Description 
PreSortData 
NO 
Sort raw event data
(btwf2) by input values first 
StockTradeAmount 
100000$ 
For Stocks: Standard
Trading Size in $ 
WaitSec 
30 
Number of seconds to wait
before the Automated Conversion starts after the TS Optimization has finished 
EventScan 
YES 
Perform a quick scan thru
all events to determine the type of position change which can occur 
MaxPosKillLookback 
100 
Maximum number of
positions per permutation to lookback during the initial data Conversion 
MaxPosShow 
10000 
Maximum number of
positions allowed per Trade Permutation 
MaxPosEqualCheck 
100000 
Maximum number of
positions when the CPU intensive position duplication check is turned OFF 
MaxEvtLoad 
20000000 
Maximum number of Events
to load (more will be truncated) 
SizeConfirm 
100000000 
Filesize (in bytes) to
confirm loading (0=no confirm) 
StockUseTrdAmt 
YES 
For Stocks: Ignore input
volume and calculate size for fixed standard trading size (see
StockTradeAmount) 
ExportEvents 
NO 
During the initial
conversion (Event > Position) 
EvtExportMin 
0 
Starting Event Sequence
Number (0=first Event) 
EvtExportCnt 
1000 
Number of Events to be
exported (from EvtExportMin) 
SkipInp 
0 
Number of Input Parameters
to be skipped (rarelly used) 
In this section you can
adjust settings related to the Position Exit Manager (PEM), which is used in
Potential Mode during the exit optimization:
Name 
Default 
Description 
MaxPEMOpt 
10 
Maxmimum number of
Position Exit Manager Optimizations to show directly in the Treeview 
MaxExitResults 
5000 
Maximum number of Position
Exit Results shown in the TreeView 
MininumExitPosCount 
2 
Set the minimum number of
positions required for the Position Exit Mgr Result List 
PEMUpDownSteps 
3 
Number of up and down
steps in creating a definition around the optimum 
PEMVerbose 
0 
Set the level of
diagnostic messages from the Position ExitMgr in the log file 
In this section you can
adjust settings related to the general BTWFMgr behavior:
Name 
Default 
Description 
OpenLastFile 
YES 
YES=On startup open the
last BTWFMgr data file (default) 
CsvViewer 
CsvViewer.exe 
Designate the default CSV
File Viewer program location 
DRIVE 
C: 
Designate the main drive
(with colon!) were all BTWFMgr data and programs resides. 
StartingCapital 
10000$ 
Define the initial
Equity/Capital for the DrawDown Percent calculations (Min $10000) 
ExportTrdPerm 
NO 
YES=Export all
Trading/Potential Permutations to a tabbed text file(*.TAB) and them imports
to a MS Access Databse(.MDB). 
MinFreeRAM 
50.0MB 
Minimum free Virtual
Memory while processing the initial data  if less abort the conversion and
continue with the available data 
ShowProgress 
YES 
Display the progress bar
window while a processes is running 
ShowProgressLog 
NO 
Write each progress bar
window message also to the log (great for diagnostics) 
LogFileAppend 
YES 
Append new message to
existing daily log file 
LogFileFlushOnEach 
NO 
Write each log mssage
physically to disk 
RefreshFileAssociations 
NO 
Reinitialize(YES) the
connection between the BTWFMgr files and BTWFMgr 
WelcomePend 
NO 
Show Welcome and initially
import the EasyLanguage Logic is into TradeStation 
FileMode 
0 
File Caching Mode (0=in
RAM(recommended), 1=on Disk, 2=no caching(slow) 
FastExit 
YES 
Use fast exit method (dont
free each allocated memory fragment) 



In this section you can
adjust settings related to the Chart settings:
Name 
Default 
Description 
ShowOneDay 
NO 
Stop the Position Bar
Chart display when the day ends 
LineWidth 
2 
Thickness of lines in the
chart (in pixels) 
MaxGridLines 
20 
Max Number of X Grid Lines
in Equity and Potential Graphs 
AxisMarkX 
4 
Horizontal Axis (X) grid
marker length in pixels 
ZeroWidth 
4 
Width of zero line in
pixels 
BBWidth 
2 
Width of the Bollinger
Line in pixels 
BBStdDev 
2.0 
In the Bollinger Band, set
the Standard Deviations 
BBAvgLen 
10 
Bollinger Moving Average
Length 
BBColorDev 
Orange 
Bollinger Line Color 
BBColorAvg 
Magenta 
Bollinger Average Line
Color 
EquMedColor 
Blue 
Straight Medium Equity
Line Color 
EquMedWidth 
3 
Straight Medium Equity
Line width in pixels 
BarsBeforeEntry 
10 
Number of bars to show
BEFORE the position entry bar in the Position View Chart 
BarsAfterEntry 
100 
Number of bars to show
AFTER the position entry bar in the Position View Chart 
InitBarDisplay 
D1/30 15:20 
In the Bar Data Display
Mode define the Starting point for the Chart display  Two formats are available: 
Download the current free BTWFMgr trial copy from:
www.profsoftware.com/bt/dl.htm
The internet browser will show
the download box:
Click on the RUN button
You will see the progress of the download …
When the download has completed – confirm the BTWFMgr installation: Click on
the RUN button
After a few seconds the actual BTWFMgr installation will start: Click on the
NEXT button
Click the NEXT button:
Click the NEXT button:
Click again the NEXT button
and the BTWFMgr will be finally installed … showing you the progress
when the installation is completed – click on FINISH to start the BTWFMgr with
the sample data
While the sample data is
installed the progress window will appear:
And after a few more
progress messages you will see:
click OK to start the import of the BTWFMgr function into TradeStation:
While you see the initial
main BTWFMgr window – TradeStation will start the Import Wizard;
you might have to login to TradeStation first.
Click on NEXT and follow the instructions:
After the BTWFMgr function
have been imported into TradeStation– return to the main BTWFMgr display
Clicking on the yellow icon
at the bottom of your desktop – in the task list:
This will reactivate main BTWFMgr Window – showing you the Equity Graph of the
Sample File:
Now you are ready to use
BTWFMgr.
BTWFMgr 
Diamond Backtest
and Walk Forward Manager 
Equ(Equity) 
BTWFMgr “Equity”
mode to detect the best performance for a strategy 
Entry Point 
The point in time
(i.e. a bar on the chart) 
Equity
Graph/Curve 
A graph showing
the accumulated equity of a sequence of positions 
Exit Point 
The point in time
(like a bar on the chart) 
Gross Profit/Loss 
The raw profit or
loss of a position 
Net Profit/Loss 
The actual profit
or loss of a position after the fees/fees are deducted 
PL 
Profit/Loss 
Position 
A position is a
completed buy/sell pair(Long Position) or short/cover pair (Short Position),
which results in a specific “gross profit/loss” and “net profit/loss” minus
the commission/fees. 
Potential/Pot 
BTWFMgr
“Potential” mode to detect the best entry point for a strategy 
Strategy Entry
Logic 
Logic which
determines when the strategy 
Strategy Exit
Logic 
Logic which
determines when the strategy is closing a position (sell/cover) 
Strategy Input Graph 
A chart showing the average “Strategy Potential” for each input value 
Strategy Input
Parameter 
A list of
variables which define the current actual values of parameters 
Strategy Input
Permutation 
A specific
combination of strategy input parameters 
Strategy
Potential 
The average
profit/loss of all entry points the strategy produced for a given
permutation, symbol and timeframe – plotted with elapsed time from entry 
Strategy
Potential Graph 
A chart showing
how the “Strategy Potential” develops over time/bars 
Strategy Preparation 
The automated process of adding a
small piece of code at the end of your strategy logic for the BTWFMgr data
collection. 
Trade Frequency
Graph 
A chart showing
the number of trades(long/short/all) for each parameter 
Trading Strategy 
logical steps
which produce “Entry Points”(Buy/Short) and 
WFA 
WFA = Walk
Forward Analysis 
Walk Forward
Analysis 
a step by step
verification how predictive/robust the strategy performance is 
(Last
Update Nov 10th, 2014)