Getting EOD data into Amibroker (www.float.com.au)?

[markrmau]

Hi all,

What is the best way of getting EOD data into amibroker?

I have tried to download metastock data from http://www.float.com.au/ but I only seem to get csv data, not metastock, and amibroker complains that the data has $0 for some close prices.

I have selected metastock from this link
http://www.float.com.au/scgi-bin/prod/dldata.cgi?month=200412

Where does the MASTER file that amibroker wants come from?

Thanks,Mark.

[RodC]

hi mark,

I've been downloading the daily zip file from www.float.com.au extracting it and then importing into amibroker. It's been working fine.

I don't know what you mean by master file.

regards,

Rod.

[markrmau]

Master seems to be needed when you specifically import metastock format.

I seem to have done it now. I was concerned that I would have to import each day individually (which would take a while to do a few years for backtesting). However, I can download 2004 from the link and import them in 1 go.

[GreatPig]

Markrmau,

Sounds like you have it sorted out, but I download FCharts compatible data and load that in. I also did my own format file (just a slight modification of the default one) as follows:

$FORMAT Ticker, Date_YMD, Open, High, Low, Close, Volume, Group
$SKIPLINES 0
$SEPARATOR ,
$CONT 1
$AUTOADD 1
$DEBUG 1
$NOQUOTES 1

I don't have any problems using this file. Format files are in the Formats folder under the AmiBroker root folder.

Ignore the "Group" field on the end of the format line though, as I've written a utility to modify the price files before importing to add a group number to the end of each line based on the ASX code. This lets me sort them into alphabet groups. Takes more effort though, as I have to run the program over the file before importing each day.

Cheers,
GP

[RodC]

Good to see you sorted it out Mark,

I downloaded back to about 1998 so I could back test. Just beware that some of the older www.float.com.au data is a bit dodgy when it comes to stock splits etc.

Rod.

[markrmau]

How do you get other data formats?

It doesn't matter what I select from Step 1, when I select a link under Step 2, I always get a zip file with comma separated values (csv). I was hoping to get 1 file for say 5 years data that would import in one go.

http://www.float.com.au/scgi-bin/prod/dl.cgi

Or are metastock and fcharts simply csv?

Thanks, Mark.

[GreatPig]

Markrmau,

Metastock has a proprietary binary format and an ASCII CSV format. Other than that I don't know anything about them, as I don't use them.

With FCharts CSV files, you can import a whole pile at once by multi-selecting the files from the import Open File window in AmiBroker. And float.com.au has ZIP files with a whole year's data in one file. Just download the ZIP files for each year, unzip them into the same folder, and then import the lot in one hit. It will take a little time to import them all though.

Cheers,
GP

[darian]

Hi all,
I've downloaded the trial version of amibroker but I can't seem to load the data into the software. Every time i try to load data from float.com.au, i always get this message-
"cannot write or format to defition file, check read only flags". i have tried checking the read only box but it doesn't work. Please help.

Darian

[GreatPig]

Darian,

You mightn't be using the correct format file for importing. Try the attached one.

Copy it into the Formats folder under AmiBroker and then rename it to get rid of the .TXT (I had to add that to upload it here). Its name should just be "FloatComAu.format".

Then use "File->Import ASCII" and select it in the file types before importing the price file.

Cheers,
GP

[darian]

Thanks GP, you're a champ! ;)

Just wondering what is that error report after the download? does that mean the data is not clean and as a result will not be able to see some stocks?

TA
Darian

[kaveman]

It sounds like the files you are trying to update to are Read Only as in windows read only file attributes. This could be the format files in AB that set the data arrangemetn you are trying to import.
Can you actually save anything in AB trial copy?

[GreatPig]

Can you actually save anything in AB trial copy?
You couldn't when I trialled version 4.6.

GP

[kaveman]

definitely could not when I trialled V3.90

[darian]

no, can't save anything, have to load data in everytime use it.

Darian

[darian]

Hi all,
I've got two computers at home and would like to install amibroker on both machines once i register. Is that a problem? Is it restricted to per computer use?

Cheers
Darian

[captain black]

Hi all,
I've got two computers at home and would like to install amibroker on both machines once i register. Is that a problem? Is it restricted to per computer use?


Having AB on 2 PC's is fine. Just need to copy the registration key onto both installations.

[darian]

thanks cap!

[Slick Rick]

Hi All,

I recently downloaded the trial version of Amibroker (v 4.90), but can't seem to load historical ASX data into the program.

2005-2007 EOD Data was downloaded from www.float.com.au and has been extracted into one folder. The data is daily files in csv format.

Using the import wizard, the fields are defined as;
Ticker, YMD, Open, High, Low, Close, Volume

Seperator is Comma(,), Group 255, Log Errors, Auto add new symbols, allow negative prices.

When I select a symbol after the data has been loaded, the graph is blank and says "Not enough data available. To plot chart 3 bars are needed, 0 bars in 'XXX'."

Not sure what I'm doing wrong?

Most grateful for any help.

[R0n1n]

I could be wrong, but the trial version dosent save data.

[Slick Rick]

Yeh, I don't think the trial does save data either.

So, that means it's not possible to load any external data into the program until you've registered?

[GreatPig]

No, you should be able to load data into AmiBroker and play with it, just not save it to the database.

I don't use the import wizard, so can't comment on that. Instead I use a format file detailed in a message up-thread and just use the Import ASCII function on the file menu, selecting my format file.

GP

[kaveman]

You can load it, but you are limited to the number of symbols, I believe
When you close Ab you will lose any data loaded (no save)

[Investor100]

Hi

I also use EOD data from www.float.com.au

I download the latest.txt file and save as for example 20070706.txt on my computer and then import into amibroker

Does anyone have any idea how to automate the process? Maybe using jscript? ie click on script and the file latest.txt then save on the computer as in "today" format as YYYYMMDD.

Thanks

Happy Trading

[theasxgorilla]

I use the data I'm fed from Market Analyst and it exports that data as CSV into alphabetical folders A, B, C etc. I've written a wscript (???, i think thats what the language was called, like vbscript) that compares the CSV files in each folder with a filter list (ASX300) and then pushes them into the one folder. From there I can use the import wizard to suck them up into Amibroker, but that last step is a manual process. I don't know of a way to automate it.

[GreatPig]

I don't understand what you're trying to achieve. Importing a CSV file into AmiBroker is a one-step process, assuming you already have a formats file and are not using the wizard each time. I don't see how using a script would be any quicker or easier.

Cheers,
GP

[Investor100]

The automate part : download lastest.txt file and rename it as today date in this format YYYYMMDD ... then I can use wizard to import it.

I like to keep database in daily raw data for reference by date.

The automate allow me to schedule windows to do daily download of the data and rename for me accordingly..

thanks

[Chorlton]

I use the data I'm fed from Market Analyst and it exports that data as CSV into alphabetical folders A, B, C etc. I've written a wscript (???, i think thats what the language was called, like vbscript) that compares the CSV files in each folder with a filter list (ASX300) and then pushes them into the one folder. From there I can use the import wizard to suck them up into Amibroker, but that last step is a manual process. I don't know of a way to automate it.

Hi Gorilla,

Could your wscript be modified so as to compare metastock files (.dat format) rather than the CSV ones?

The reason I ask is that I also have my data in alphabetical folders and would like to perform a similar task.

Regards,

Chorlton

[bOsO]

Man you're doing it the wrong way, I can give yo a database that you can download 20min delayed data from yahoo or historical also from yahoo I also went trough the hustle of finding out how to do it

also I just bought version 5.0 and get real time from FXCM


Hi all,

What is the best way of getting EOD data into amibroker?

I have tried to download metastock data from http://www.float.com.au/ but I only seem to get csv data, not metastock, and amibroker complains that the data has $0 for some close prices.

I have selected metastock from this link
http://www.float.com.au/scgi-bin/prod/dldata.cgi?month=200412

Where does the MASTER file that amibroker wants come from?

Thanks,Mark.

[bOsO]

I can give yo a database that you can update amibroker with 20 min delay data from yahoo of all the stocks also historical data
send ma an email boso400@gmail.com

[kolonel]

Digging up an old thread, but i take it you pay for what you get in regards to data ?

Being a free service, there must be some portion to it that is missing ?

Been a while since i frequented these forums, but catching up on some reading.

Regards
kolonel

[howardbandy]

Greetings all --

Here is a link that might help explain getting data into AmiBroker:
http://www.introductiontoamibroker.com/book.html

Scroll down a little and click "Installation". You will be getting a pdf file that is Chapter 2 of my book, "Introduction to AmiBroker". (Several hundred copies have already been sent to customers in Australia.) Note that one of the subscription data services covered is Norgate Premium Data. They have excellent coverage of the Australian market, as well as most of the rest of the world.

When it comes to data for AmiBroker, or any other charting or trading system development package, ask yourself whether you want to be in the data maintenance business or the trading business. I think Norgate and the other data vendors provide an excellent product at a reasonable price, and their efforts to keep track of data errors, splits, and so forth free me to concentrate on other things.

Thanks,
Howard

[pennies]

Hi

I hope someone can help me. Ive been using FCharts and Quotetracker for a year or so now. I want a better Charting package so I am trialing Amibroker. When I used FCharts I could click "Import/Export" and it would automatically update all of the data for the ASX & US stocks in my portfolio. Can Amibroker do that without the need to use Amiquote. Can I just tell Amibroker to download from Yahoo somehow? Can Amiquote also down load intraday data?

Thanks:banghead:

[howardbandy]

Hi Pennies --

I am not familiar with FCharts or QuoteTracker, so cannot make a detailed comparison. But the trial version of AmiBroker has all of the charting features enabled, and the installation procedure installs a small end-of-day database, so you can make your own comparison.

AmiBroker uses AmiQuote to retrieve data from Yahoo or msn and store it in AmiBroker. It works seamlessly, so there is no disadvantage in using AmiQuote. After you have set it up once, just use the Tools menu, then click Auto update quotes, and AmiBroker will handle the interface with AmiQuote and Yahoo automatically. AmiQuote runs, but you will not need to run AmiQuote yourself.

But ----- think about whether you want to be in the database maintenance business or the trading business. Yahoo quotes are cost-free, but not trouble-free. You will find missing data, inconsistent data, unadjusted splits, and so forth. I strongly recommend two things related to data:
1. Consistency is important. After you have tried out several data sources and found the one that provides the data series you want, build your historical database using that source and always get your daily updates from them. Do not mix data from different sources.
2. Try out the subscription data sources, then pick one and hire them to be in the database maintenance business for you. If you will be following the Australian markets, Norgate Premium Data is one you should check for sure. Norgate also has data for United States, Singapore, etc. Norgate is end-of-day. Also check Quotes Plus if you need only the US markets. Quotes Plus has both end-of-day and intraday.

If you will be following your positions intra-day, but not using intra-day bars (say, 5-minute bars) to feed your trading systems, you can use one of the many web sites that give real-time snapshot quotations.

If you will be using intra-day bars for your trading systems, subscribe to a good vendor of intra-day data. You cannot run intra-day systems using free data.

There are several other factors that make using intra-day data more complicated than using end-of-day data. If you are new to trading system design, testing, validation, and trading, practice using end-of-day data and end-of-day systems before moving to intra-day.

(No disrespect intended. You may be more experienced than I am, but I do not recognize your user name.)

Thanks for listening,
Howard

[pennies]

Thanks Howardbandy.

Thank you so much for the time that you took to reply !!

At this stage I mainly check charts at the end of the day but I do use it for intraday occaisionally. At the moment I dont have access to "live" data but I use quotetracker to update the charts on a 5 second time delay which is fine for what I do. One great feature the FCharts has is it will backfill intraday data. Do you know if Amibroker will do this?

[howardbandy]

Hi Pennies --

AmiBroker has the capability to backfill intraday data. It depends on the source of the data whether it will backfill or not. Data coming from streaming data quotes, such as QuotesPlus, eSignal, DTNIQ, and so forth will backfill. Data coming from snapshots, such as Yahoo or your brokerage account quote page, will not. Data coming from trading platforms usually do not -- I know that Interactive Brokers does not cause a backfill, I do not know about FCharts. The easiest way to find out is to try it. Then let us all know.

Thanks,
Howard

[julius]

Hi Everyone,

You can backfill with Interactive Brokers, but only back to about 1 month using 1 minute data. I've found that if you use a resolution below 1 minute (5 sec, etc), it tends to be glitchy.

[howardbandy]

Hi Julius --

I use Interactive Brokers as a trading platform and like them. I use their real-time data during the trading day, but I do not use their data to store in my real-time database for use in trading system development.

If you plan to build intraday bars, I recommend using a vendor of real-time quotes rather than / other than Interactive Brokers. IB sends data for only a portion of all trades.

Thanks,
Howard