He's the author of: Design, Testing, and Optimization of Trading Systems
Well worth a read.
He's the author of: Design, Testing, and Optimization of Trading Systems
Well worth a read.
The bottom of page.7.Is your approach totally mechanical?
Yes. The only thing thatís not mechanical is when we do the re-optimizations each year; thereís human evaluation of the results which is not done mechanically.
Any thoughts on re-optimisations?
Not sure who the interviewer is but he's asking all the right questions.
Curtis Faith (in his book) also mentions that we should accept that maxDD be double that which was encountered in testing. ie. if we get maxDD of 25% in testing, be ready to accept 50% when trading live.How do you differentiate a large but acceptable losing streak from one that negates your perceived boundaries? Have you ever had one go beyond accepted parameters?
Yeah. The day that the U.S. said diplomacy is dead in the Iraqi situation we were long bonds everywhere, short stock indices everywhere, long energy and short the dollar. We had the worst day in our trading history. We just decided that that’s what happens once in a while. We didn’t feel there was anything out of the ordinary.
Here’s what I think the primary consideration is. Let’s assume you did everything correctly. You tested on as large a data sample as you had available, and you came up with your max drawdown. Even going way back, a lot of people maintain that what you should do is, assume that your max drawdown is understated. Assume that it’s actually
going to be at least twice as bad. Certainly in our S&P models, drawdowns have gotten bigger over time, but then, volatility has also increased. The way you distinguish whether the increase is acceptable is, the current drawdown verses the current volatility should be somewhat proportionate to the original drawdown and the original volatility. If the volatility and drawdown both doubled in size, your profit should also approximately double. If that’s what you have, that’s ok in my book.
If your profits remain the same, volatility has not really changed, and you suddenly have a drawdown that’s twice your original one, it doesn’t mean the model has failed, but it does mean you’d better find out why. It’s a potential red flag .
THis article is GOLD.
His book must be a winner as well.
Bob Pardo's book has a 1992 copyright date. I'd recommend borrowing it from a library before spending the money to buy it. You'll note it is not included in my list of recommended reading:
I'd recommend the books by Van Tharp and David Aronson.
In 1998 Bob called me looking for a job.
Aussie Stock Forums 09-05-2007
Thanks for your kind words.
Art Collins (http://www.pardocapital.com/pgl_consult_collins.html) is the interviewer of the interview with me to which you posted a link. It is the lead interview in his very fine -- though rather poorly titled -- book Market Beaters. Art, by the way, also makes his primary living from trading. He is also the author of three other excellent books http://www.pardocapital.com/pbl_team_collins.html.
It may be of interest to note that -- the opinions of Mr. Bandy and Mr. Tharp aside -- DTOTS is still considered one of -- if not the -- best and most thorough introductions and manual for the testing and optimization of a trading system. Additional information about DTOTS is available at our website http://www.pardocapital.com/pbl_pardo_dtots.html.
It also may be of interest to note that an extensively updated and expanded 2nd edition will be available 02-2008 http://www.wiley.com/WileyCDA/WileyT...470128011.html.
It may also be of interest to know that I am a professional trader -- also a consultant, coach, mentor and software developer -- and commodity trading advisor and have been such since 1996 (sorry Howard, I have no recollection of ever contacting you for a job, let alone in 1998, although I do talk to a lot of people about a lot of different things . . .) For additional information see http://www.pardocapital.com/pcl_index.html. I have enjoyed a strategic alliance with Dunn Capital Management since 1996.
For those who think the techniques in the book are "outdated" it may be of interest to know that our current trading program XT99 has posted a return of +305% (http://www.pardocapital.com/files/XT...%20Barclay.pdf) as of the end of July and since inception. As of July 2007, XT99 was +37.5% YTD. Preliminary estimates for August are +11%.
In addition, XT99 has been fortunate to have been cited over 25 times (http://www.pardocapital.com/pcl_rank_all.html) for top performance by Barclay and Futures magazine.
XT99 was originally created using the techniques presented in DTOTS.
PCL still uses the basic techniques in the book. Of course, we have added a few things since then. I am the inventor of Walk-Forward Analysis (which is trademarked by PGL) and which is one of the main focal points of DTOTS and EOTS (the 2nd edition.) I would not trade any strategy which was not tested in that manner. Nor will PCL.
By the way Van Tharp, since you made the comment that DTOTS is "not really related to systems testing and design,Ē I guess you might want to consider giving it a reread for that is ALL that it is about. Just Google the name of the book or my name and see what everybody else says about it . . .
Discussion only! Posts may be factually incorrect due to ignorance, taken out of context, misinterpreted, or just opinionated discussion.
Hi Bob --
Thanks for your responses.
I am looking forward to your new book.
While in graduate school, I published a paper on walk-forward testing, and called the process exactly "walk-forward", in about 1969 or 1970. I just got through looking through my files for a copy, but didn't find it. I don't claim to have invented the process or the name, just a long history of working with modeling and simulation.
If you had a decent system and you wanted to make 50 grand a year, you probably need to trade with between 250 and 500 thousand dollars. If youíre going to make 10 percent a year, youíre doing pretty good. Fifty thousand off $250,000 is 20 percent a year. My compound rate of return for the last three years is 40 percent, which makes me the fourth highest performing advisor for the past three years.
People hear stories about a floor trader who makes $300,000 a year trading with maybe $100,000. They figure, "well, if he can do it, so can I." Itís not the same thing. Floor traders can leverage their clearing houses in a way that isnít possible for the average person trading as an outside retail customer. .
Bob, I am very interested in your comments on this particular answer in your interview.
I don't particular agree with it because with the advancement in technologies and leverage capability, one could yield a much higher "absolute" return system (though reward/risk ratio may still be realistic) through trading multiple markets using extremely short intra-day data, thereby creating far many more opportunities.
Back in the old days, i guess this is impossible for one person because he/she would be physically impossible to monitor and trade such many opportunities.
But nowdays, people have developed fully automatic adapative intraday trading systems, which is statistically proven with valid backtesting/optimising methods, and trade them on large number of markets that gives them lots of trading opportunities. Since the execution itself is fully automatic, the markets can be traded 24 hours a day as well, adding to the overall profit edge.
I have seen evidences (though not through personal experiences YET) of such possibilities.
Regardless, your book has been on my wishlist for several weeks now. Really looking forward to order it, but you say there is a second edition coming up on 2/2008. Don't know if i can wait that long though.
Iv finished reading this book today.
I must say i would highly recommend this to anybody serious about system testing and design.
Looking forward to the sequel.
I would like to read your paper on walk-forward testing if you are ever able to locate it.
I would also like to read your book if you would be so kind as to forward a copy to me.
Regarding rates of return, I agree with you.
In the context of the interview, Art was asking what it takes for the "typical" or "average" trader to earn that money.
There is a section in EOTS which discusses Perfect Profit -- a concept which I invented and first formulated in DTOTS -- for different markets and time frames and the vast profit potential which this suggests.
There is also an extensive section which discusses how trading strategies and strategy development software has really lagged far behind current computer and communications technology.
Yet, even that being said, true rocket scientists like Jim Simons and David Shaw, while leading the pack in returns for hedge funds, still are falling far short of what is theoretically possible.
In EOTS, I have tried to raise the bar of returns expectations considerably. I hope to prove the point myself with new projects for PCL.
And, by the way, XT99 was +11.86% for 08-2007, +53.78% 2007 YTD and +354.03% since Inception.
Preliminary estimates are +14% for 09-2007.