Backtesting - a few questions

[Twiddle]

Hi all,

Any help answering the following would be greatly appreciated.

1.) For backtesting intraday stategies, I assumed you need historical data with minute closes. Is this correct?

2.) Having data with detail down to the minute must result in some fairly large files? Does this mean backtesting can take a very long time to process depending on the time range and complexity of the strategy?

3.) Is it easy to obtain historical data cheaply, or do you have to have a data feed or account (for example with IB) to be able to access the required data for effective backtesting?

4.) When running a backtest for intraday strategies, how accurate will would the results be with the exact data, but real time... liquidity problems, orders not filling in time etc.


Basically my situation is; I have downloaded ninja trader and would like to start designing systems, however I really don't know where to start with obtaining the data. To be honest it is a bit daunting as a complete beginner.

If it comes down to having to buy that data I am ok with that, but what kind of costs would I be looking at?

 

[waza1960]

1.) For backtesting intraday stategies, I assumed you need historical data with minute closes. Is this correct?

Yes generally except for tick data and I wouldn't worry about that if your starting out.

2.) Having data with detail down to the minute must result in some fairly large files? Does this mean backtesting can take a very long time to process depending on the time range and complexity of the strategy?

Gain have historical minute data, 1 year covering most fx pairs is approx 600mb.
However when you backtest on ninja the data is only downloaded for the pair you are testing at the time and IMO this is insignificant.
I can backtest a relatively advanced intraday strategy over two years in less than a minute with ninja.

3.) Is it easy to obtain historical data cheaply, or do you have to have a data feed or account (for example with IB) to be able to access the required data for effective backtesting?

You can get gain fx data for free (two years on common pairs and less on others)just go to the broker links on the ninja website .The next step for fx woud be to open an account with gain $300 minumum to get full historical data or you can get free emini data for one month through other brokers.

4.) When running a backtest for intraday strategies, how accurate will would the results be with the exact data, but real time... liquidity problems, orders not filling in time etc.

Well thats why you go into simulation first to help quantify these issues keep in mind that even simulation results can vary to real trading but it is IMO an important and valuable step that must be taken.
In my limited simulation experience there can be more than10% difference between backtested results and simulation results.
I wouldn't pay for any data to start with I would familiarize myself with ninja using free data first.

 

[tech/a]

Hi all,

Any help answering the following would be greatly appreciated.

1.) For backtesting intraday stategies, I assumed you need historical data with minute closes. Is this correct?

Yes if trading 1 min timeframe or any increment you wish to trade.

2.) Having data with detail down to the minute must result in some fairly large files? Does this mean backtesting can take a very long time to process depending on the time range and complexity of the strategy?

More about computer and software capability.

3.) Is it easy to obtain historical data cheaply, or do you have to have a data feed or account (for example with IB) to be able to access the required data for effective backtesting?

Depends on what you want but it is very expensive.----can be $1000s

4.) When running a backtest for intraday strategies, how accurate will would the results be with the exact data, but real time... liquidity problems, orders not filling in time etc.

As good as the developer and software.Slippage and liquidity in some markets will occur. Best avoid illiquid instruments plenty of liquid ones.

Basically my situation is; I have downloaded ninja trader and would like to start designing systems, however I really don't know where to start with obtaining the data. To be honest it is a bit daunting as a complete beginner.

Can be took me 2 yrs to get my first and can be addictive.
You learn more when developing systems I believe than at any other time.

If it comes down to having to buy that data I am ok with that, but what kind of costs would I be looking at?

See above.

 

[Wysiwyg]

You learn more when developing systems I believe than at any other time.

A gem.

 

[Twiddle]

Thanks for your help all.

I think I will start with getting the gain FX data and work to familiarize myself with the whole process.

Tech/A - from my experience so far my preferred things to trade are Crude, DAX index, and EUR/USD - are these liquid enough for reasonably accurate backtesting results?

 

[waza1960]

Gain will only get you access to FX pairs .
EUR-USD IMO seems to be the hardest pair to make good systems on in line with my belief that the most heavily automated traded instruments are the hardest to trade because they are aguably the most efficient especially with FX.
I can't wait to try commodities and indexes in the future probably through IB I think they will suit system development.
Of course to put my views in context Tech/A is the guru I'm still on the path to enlightenment, guru status whatever.

 

[Trembling Hand]

Tech/A - from my experience so far my preferred things to trade are Crude, DAX index, and EUR/USD - are these liquid enough for reasonably accurate backtesting results?

He wouldn't know but that won't stop him giving you a misleadingly authoritative answer !!

 

[waza1960]

Well TH you made me lol when I'm not the laughing type.
Sorry it was at Tech's expense though .

 

[Trembling Hand]

Well TH you made me lol when I'm not the laughing type.

Glad to help out wherever i can.

Be very careful backtesting on 24hr instruments. Spreads & slippage can be very wild outside of cash hours.

Also when you get down to backtesting off 1 min data on the CL or 6E for example you can forget about getting accurate fills and slippage, especially on stops. Although they a very liquid contracts during cash hours sweeps rain supreme.

Then of course you also have the rollover/expiry problem to be mindful of with your data.

Best to not be too excited with intraday results on small time scale until you can confirm in forward testing/live testing.

 

[tech/a]

Oh me?? sorry bit busy at Board meetings!

Not trading futures (Just the SPI) but yes they would be liquid enough.

Far better than stocks-- re liquidity.

Here's one for the fans.

 

[Twiddle]

Glad to help out wherever i can.

Be very careful backtesting on 24hr instruments. Spreads & slippage can be very wild outside of cash hours.

Also when you get down to backtesting off 1 min data on the CL or 6E for example you can forget about getting accurate fills and slippage, especially on stops. Although they a very liquid contracts during cash hours sweeps rain supreme.

Then of course you also have the rollover/expiry problem to be mindful of with your data.

Best to not be too excited with intraday results on small time scale until you can confirm in forward testing/live testing.

OK, I am starting to see why I thought short time scale trading is easier than it is. When trading in the sim all of my positions were filled instantly, making it extremely easy to get in and out of very small trends quickly.

Is it possible to make a lot of very short time scale trades, say 30 seconds to 2 minutes, 30 trades and hour or will fill problems etc stand in the way of this type of trading, and I should be looking to another style?

If it is possible, I am guessing designing systems with backtesting would need to be done with tick data.... is that possible?

To give you an example of the type of trading I would like to carry out; crude quite often jumps up say 10 - 15 ticks in a short time, away from its underlying trend, then very quickly declines back to more inline with the trend. I like jumping in as soon as the candlestick appears to settle and open a position in the opposite direction, and limit the order to 3 points higher than the spread. Small profit, but it only lasts say 1 minute, rinse and repeat. Ideally I would like to design a system that can capture these types of things, so I do not have to be watching multiple charts like a hawk constantly... Is this doable, or am I wasting my time with such short term trades?

 

[Twiddle]

Oh me?? sorry bit busy at Board meetings!

Not trading futures (Just the SPI) but yes they would be liquid enough.

Far better than stocks-- re liquidity.

Here's one for the fans.

Cheers Tech/A

Very nice pic.

 

[Trembling Hand]

Ideally I would like to design a system that can capture these types of things, so I do not have to be watching multiple charts like a hawk constantly... Is this doable, or am I wasting my time with such short term trades?

Give it a go. My guess is it will be far harder than you have described by x100

But for starters 1 min data will be useless. For that time frame you need Bid/Ask prices & volume in tick format and will need to run the app on a virtual server in NYC because the latency from Oz to the US is about 1 sec round trip.

Your competing with the Bil dollar Bots at that time frame. Good luck

 

[Twiddle]

Give it a go. My guess is it will be far harder than you have described by x100

But for starters 1 min data will be useless. For that time frame you need Bid/Ask prices & volume in tick format and will need to run the app on a virtual server in NYC because the latency from Oz to the US is about 1 sec round trip.

Your competing with the Bil dollar Bots at that time frame. Good luck

OK, so it looks like that type of trading is out of the question.

I am still very much attracted to short term trading... from looking at your scalping threads in the past it looks like you are probably the guru of quick trading around these parts TH... do you have any advice for a newbie like me of where to start for intraday?

 

[Trembling Hand]

Definitely not at the scalping end more misery there than opportunity. There is huge opportunities from 5 min out to hourlies without the churn cost and execution requirements.

 

[Twiddle]

Cheers for the pointer.

 

[tech/a]

You still scalping T/H?

Twiddle

You maybe interested in having a look at some of these.
http://www.google.com.au/search?hl=...vid:1&q=VSA+CRUDE&aq=f&aqi=&aql=&oq=&gs_rfai=

 

[Trembling Hand]

You still scalping T/H?

About 50 trades per day on the HSI (main instrument). Not what I would call true scalping but may fall into that category for the uneducated.

 

[tech/a]

About 50 trades per day on the HSI (main instrument). Not what I would call true scalping but may fall into that category for the uneducated.

Not a lot different from your Scalping thread then? (Trades/session)
Guess you were un educated back then.
There is a T/H measure for Number of trades/session for the educated to benchmark from? Strangly I could not find it on google.

 

[Trembling Hand]

Not a lot different from your Scalping thread then? (Trades/session)
Guess you were un educated back then.
There is a T/H measure for Number of trades/session for the educated to benchmark from? Strangly I could not find it on google.

the SPI hedges were for about 1 hour a day if you bothered to think about it.

The HSI trading is over 4 3/4 hours you do the maths.

Scalping is scalping. You clearly don't know what that is.