- Joined
- 12 November 2007
- Posts
- 1,629
- Reactions
- 47
Hi TM --Why does a time series make it more difficult for machine learning? I don't really understand how they're teaching machine learning/AI to play video games or board games and it's nailing it after running overnight on thousands/millions of tries and by morning its better than any human, why can't that be done with trading, load as much history into it as you can and treat each day as a "level"(obviously intra-day data would be optimal) and let it go thousands or millions of times learning the different days, I know each day is different in trading but surely it would recognise patterns that we might not have noticed before or have a fair idea of what to expect from the training set?
Hi TM --
Machine learning, as we will be using it, is a two step process.
1. Examine some data, looking for patterns that precede profitable trades. Fit a model to the data in such a way that the patterns are identified. For example, each day at the close of trading, identify whether the next day's close will be higher or lower.
2. Verify that the patterns persist. That they continue to exist in data that was not examined during development.
In many machine learning applications, such as determining / estimating / predicting whether a borrower will repay a loan or default, the data points are independent. Each data point is related to a single person and consists of his or her employment, residency, age, etc. We could write the data on slips of paper, one per person, put them in a big bowl, draw them out in any order, and treat them as being independent.
Time series data introduces a complication in that data points -- data for individual days -- are not independent. For one, we rely on comparisons between current data and previous data (did the RSI fall through 20?). For another, patterns that were profitable in the past may no longer be profitable. And, the data is very noisy (there may not be actual profit-indicating patterns -- it may all be noise).
So, in addition to the model fitting and validation process called for by the scientific process, the analysis of time series must take all of these extra issues into consideration. One of the concepts is "stationarity." My video, "The Importance of Being Stationary," might help:
Best regards
Howard
Fisher's Iris Data
Sepal length Sepal width Petal length Petal width Species
5.1 3.5 1.4 0.2 I. setosa
4.9 3.0 1.4 0.2 I. setosa
4.7 3.2 1.3 0.2 I. setosa
7.0 3.2 4.7 1.4 I. versicolor
6.4 3.2 4.5 1.5 I. versicolor
6.9 3.1 4.9 1.5 I. versicolor
etc
This is a Ducks explanation.
Amibroker can run systems tests on a pre defined formula,
Machine learning
Identifies patterns and learns how to improve the buy and sell signal by
Analysing data . It's a continuing process.
Systems are static.
Howard will make it sound really exciting and interesting.
Do you mean GNU Octave? It's cross platform and so it works on Mac. https://www.gnu.org/software/octave/Hi Howard,
I see you recommended the course on ML from Dr. Andrew Ng's at Stanford Open Classroom.
Unfortunately, he works with a Windows version of Octave and I work in a Mac environment.
Do you know where I can find instructions to implement Octave for Mac in a way that would work within Dr Ng's course? Or if there is a forum or other communication channels for students of those courses?
Thanks,
Luis
Thanks Indoril,Do you mean GNU Octave? It's cross platform and so it works on Mac. https://www.gnu.org/software/octave/
All good. Installed and working with the course.Oh I see. Sorry, I didn't realise.
How would you use 'Watson Analytics' by IBM?How long before there's a point-and-click retail version of machine learning software? Something like Watson's Analytics for financial markets?
I would be very skeptical about the ultimate usefulnessof such material. At best they are extreme oversimplification of the problems leaving you with a big hole in your applications and knowledge.
Thank you for your excellent series of posts in this thread!One more thing. Time series systems add a considerable complexity. They are not stationary. They are not static. They are dynamic. No matter how the system was developed (traditional platform or machine learning), there will be periods where the signal is clearly identified and can be profitably traded. And periods where it is inaccurate and is not profitable. We cannot tell whether a period of poor performance is a temporary loss of synchronization between the model and the data and will recover; or the first indications that the system is broken and will never recover. Prudent system management reduces position size in response to poor performance. The relationship between the recent trades and the safe position size that results in the highest wealth over a period of time is itself a system -- the trading management system.
Best, Howard
We use cookies and similar technologies for the following purposes:
Do you accept cookies and these technologies?
We use cookies and similar technologies for the following purposes:
Do you accept cookies and these technologies?