dataload bug - drops species when there are a large number

Submitted by Guest on Fri, 28/09/2012 - 05:51

Implementation discussion

Attached to this post are two setup xml files (renamed with .txt extensions so that they can be uploaded). They differ only in the order of the species data. setup-bad.xml has the species data in breed/species order (it starts at line 934 and ends at 2321). setup-OK.xml has the data in the same order, but with the L to Z part moved before the A to L part (again the data starts at line 934 and ends at 2322, with the order change at line 1588).

If you dataload the setup-bad.xml file, then the Breed lookup table contains the full data set. However, if you create or edit a patient and set the species to Dog or Cat, then the pull-down list of breeds is missing the A to G breeds. The other species which have far fewer breeds are OK.

If you use the setup-OK.xml file then all is good for the dogs - all 469 dog breeds are there. However, the cat breeds go from A to K - those after K are missing from the dropdown list. So my order swap did not do a full fix.

Note that before doing the dataload there is a full reset, ie the openvpms database is dropped and then recreated, and the archetypes loaded. Similarly the Tomcat service is restarted after the dataload before logging into OpenVPMS.

Version: OpenVPMS 1.5.1

When you look at the xml files you will see that they contain most of the setup data required for the practice. I have not played with stripping the species/breed data into a separate file so that it can be tested independently of the other information.

I have also attached the log of the dataload - this does not change from the OK to the bad case.

Regards, Tim

-------------------------------------

After further playing & testing, I understand what is going wrong. Its true that all the breeds are added to the lookup list. However, the problem is that that some of the species targets are missing. ie the cats and dogs 'missing' from the pull down breed selection list have no species set for them.

ie the dataload program appears to be screwing up when there are a large number of lines that refer to the same "source" - ie when you have many breeds of the same species each with entries like:

Re: dataload bug - drops species when there are a large number

Submitted by tgething (not verified) on Sat, 06/10/2012 - 14:10.

I have an avoidance procedure - don't inject so much data in any one dataload run. I my case I split the load into 5 runs: setup, postcodes, dogs, cats, other-breeds.

All now works - ie I have all the dogs and all the cats.

Only one problem - my setup data set needs a definition of the Dog and Cat species (because I am loading species specific reminder data). The 'fix' is to use the species Dog1 and Cat1 in the setup.xml, and then when all is loaded, login as Admin and replace the species Dog1 by Dog and ditto for the cats.

Regards, Tim G