August 22nd, 2010 andrew
Well we have had the election and there is no clear winner but it appears that the Rudd/Gillard government is the loser. eHealth is up for some reassessment and it certainly needs some.
From all reports $5 Billion of taxpayers money has been spent over the last 10 years and we have little to show for it. 10 years is long enough for any reasonable plan to bear fruit and there is clearly some fundamental flaw in the methodology.
The decision to purchase a SNOMED-CT license, and a Certificate Authority is the only positive I can find. We do have a Secure messaging standard, but it is flawed by the fact that it really depends on Nehta “NASH” service for suitable certificates rather than working with the existing Medicare certificates. (and NASH is vapour ware) Virtually every general practice has Medicare certificates as part of the PIP program and the reason that the secure messaging standard was not designed to work with them lies with the dysfunctional relationship between Nehta and Medicare.
Dysfunctional organisations seem to be at the heart of the matter. We have not had one organisation for 10 years, but about 4 or 5 each of which has been rebadged and restarted only to repeat the same mistakes. That mistake, or the core of it is the idea that they have to “solve” the problem and produce software. Government is hopeless at doing this, indeed most large organisations are hopeless at producing software and large projects tend to fail. What we need is government to provide governance to move the industry forward rather than trying to do the heavy lifting. After 10 years the things that were working at the start are still working and all the things that are working are based on consensus standards. What we need is governance to comply with those standards and progress the standards in an incremental way. Currently we see much talk of Nehta inventing standards and despite some very capable people inside Nehta this is doomed to failure. Standards have to be created by consensus, as then the industry will engage with the painful standards process in order to prevent silly ideas becoming a standard and to fix errors where they occur. They will only engage when they know they have a duty to comply with the standards and this is where the lack of governance is failing us.
The standards process has become orphaned because Nehta have said they will be dictating the path, and they have failed to produce any clear path. More recently they have tried to steam roll standards and this is also likely to fail as without adequate review the standards will be poor.
Out there is the real world, which is the world that Medical-Objects inhabits, we see significant advances in communication with the number one obstacle being poor standards compliance. Because of a lack of governance large vendors feel that they can flout the standard and dictate the formats though sheer market size. Because of a lack of compliance anything that does work is fragile because it is never quite right, but has to be wrong in exactly the right way in order for it to work.
The dreams of the connected health landscape are often formulated by people with no knowledge of the importance of the lower levels. We can connect to the whole world of internet services because of compliance with the invisible things like tcp/ip and http standards and not because some middle manager dreamed of the internet. The dreams of a connected eHealth world rely on applications supporting the creation and consumption of high quality, standards compliant messages and not on the glossy pdfs produced by Nehta.
I don’t think we need Nehta, the states are not the eHealth leaders in this country and Nehta was setup as an uneasy alliance of the states, many of whom ignore Nehta anyway. What we need is good governance and a focus of standards compliance of all the applications that make up the landscape. There does need to be funding of the standards process and there needs to be a mechanism for providers of healthcare to pay to buy standards compliant software, which if built properly, will be more expensive than they are paying now. However, the money they are saving is working against the big picture of a connected landscape and this is where governance and some well directed funding could make a huge difference. If every health application was required to be standards compliant we would see an enormous spike in interest in the standards process and the consensus process could be resuscitated for its premature death and we could start moving forward one level at a time.
The big bang process has failed, as it was bound to do and we need a return to proven paths. The cost would be a fraction of what was planned and the results, though slow would be much more solid. The tortoise is still in the race, its time to stop trying to follow the scatterbrain hare!
Posted in EHR, HL7, MESSAGING, SNOMED, STANDARDS | 3 Comments »
August 1st, 2010 andrew
It is interesting that so many people are keen to rubbish the encoding of HL7 V2 data. “It old and outdated” is a common comment. After a serious amount of time playing with it and experimenting with many encodings I am now convinced its brilliant.
The “xml will solve everything” mantra is fading as people realise its inefficient, and bloated and in my experimentation it works, but really offers few advantages. I use and like JSON and this is a no brainer when working with javascript. I have recently also been playing with erlang term encoding. This is used heavily in some pretty impressive scalable applications.
I have now converted clinical data into all three formats and it has made me realise the absolute brilliance of HL7 V2 encoding. Bloat with xml is a scalability issue, and the DOM structure holds up processing data as a stream, although this is not a serious issue. The human readability aspects of it is a joke. If you are going to process medical data then having it humanly readable is the last priority, especially as humans are so fallible. It is also not backward or forward compatible and no easy mechanisms exist to overcome these issues. In contrast HL7 V2 encoding, when parsed according to the rules, allows great flexibility. A field can be made repeating, or, eg. expanded from a simple string (IS) to a coded value with good backward and forward compatibility. The lack of field names is not an issue as the computer knows where the data should be. It reduces bloat significantly. It is also possible to process the data as a stream as you don’t have to wait for a closing tag. Absent values can just be omitted, further saving space. New values can be appended to the data types or segments without breaking parsers.
JSON is much better than xml, but still bloats because of field names and changing a string into an object causes significant issues. Erlang terms omit the field names, but require absent values to be encoded as blank and are inflexible when new fields are appended, making backward and forward compatibility problematic.
While free of off the shelf parsers are available for the other formats I think this up front advantage quickly fades once the other issues appear. HL7 V2 encoding does at first seem old hat, but it was developed in an age when most people in the industry were computer science gurus as the resources available were so limited, and things had to be efficient to actually run. After actually implementing transforms into the common alternatives and realising that these formats are likely to cause problems when version changes appear I have a new respect for the HL7 V2 encoding. I suspect the critics have not really used it in anger, as it just keeps on keeping on. xml might be great for a quick and dirty data exchange format where efficiency is not high on the list. Its also essential when astronaut architects are at the helm! JSON is very efficient talking to a html/javascript front end. However for the core backbone of health messaging that has to cope with multiple version of data and be efficient and not consume vast quantities of disc space I still think that well implemented HL7 V2 is simply brilliant. Despite being the poor cousin of the modern formats, its position as the most used medical messaging format does not surprise me. The tortoise might yet win the race!
Posted in EHR, HL7, MESSAGING, STANDARDS | 1 Comment »
March 30th, 2010 andrew
National Health IT programs do not have a good record of success in general, and Australia has been a good example of that to date. I don’t think anything is about to change.
The reasons for this will no doubt be well understood in time, as history looks back and shakes its head in dismay at the wasted resources and opportunities. Its hard to pinpoint the reasons for failure until you have success to contrast it with. I think a large part of the problem is the top down approach to a problem that can only be solved bottom up. By many measures Australia has been a leader in eHealth to date, but I don’t think any of that can be attributed to government policy or support as its mostly been bottom up. Certainly the National eHealth Programs of the past have failed to progress the situation and have in many ways just distorted the market for the worse.
While some will say the issue is “Change Management”, I think this is wrong. To have change management you need to have a change worth implementing and to date the quality has not been there to justify change. The quality needs to be in the software and eHealth is a complex beast to tame. To progress it we need to have the foundations to build on and currently they are sitting on swamp. Netha appears determined to adopt the tunnel vision of its “Stakeholders” ie. The state health departments and ignore the bigger picture of the international markets and standards bodies. Despite ample evidence that the existing infrastructure is cobbled together and working in the most fragile manner conceivable they want to march on and implement national programs without any compliance agenda on the horizon for at least 2 years. Building anything on the current infrastructure without a resolute compliance program is a recipe for disaster.
Australia currently has good penetration of HL7 V2 messaging, but the quality is patchy and the interoperability extremely fragile. Any change to messages results in failures and in effect we are locked into a situation where only a few systems can handle compliant data. This is interconnectivity and is a long way from interoperability, its a road to nowhere and in reality the known errors in existing lab messages cannot be corrected because of the fear of breaking existing systems. Despite Australia having compliance testing available there appears to be a complete lack of understanding of its importance by Nehta. Rather than underpin the cracking foundations before trying to renovate the building Nehta is determined to add another 3 stories to the building. The earthquake in Haiti demonstrated the dangers of a city built without adequate building regulation. Nehta’s plans will result in major loss of life at the first sign of a tremor, even if they manage to build something (which they have failed to do to date).
Interconnectivity without interoperability is a recipe for disaster and this appears to be the agenda. Delivery at all costs appears to be the political motivation and I think its time to reject the short term political goals and try and attack basic compliance and quality now. The software term “Design by Contract” was never meant to mean a business contract, but a compliance contract. Nehta appears not to get this and wants to substitute “contracts to deliver a business plan” for “contracts to comply with standards”. Computers are quite bad at being politically correct and will reject business plans that lack credibility at the binary level.
The real issue in eHealth is a lack of quality, and subsequently a lack of interoperability and safety. There are fundamental engineering deficiencies in the real world and a lack of realization that only standards, and good compliance with standards can fix the flawed foundations. Foundations are not sexy, but getting the structure out of the ground is always the biggest hurdle on a building project. To improve the situation we need a focus on good software engineering practices and in the world of complex systems that means testing and more testing. The reality is that HL7 V2 is going to be around for many years to come and rather that march on with grand plans the priority needs to be getting a compliance program for existing standards up and running now. Moving to something new is an expensive diversion that will make the problem worse, not better. Someone needs to stand up and stop claiming they will deliver the 10 story masterpiece “next year” and start work on a compliance program for what is already in use. We need some solid foundations or the Haiti style devastation of eHealth will surely descend upon us within a few years.
The Nehta plan, as it stands will deliver fragile single purpose interconnectivity with little or no interoperability. Its time we turned our existing interconnectivity into interoperability by a deliberate compliance agenda. Once thats done we will be out of the ground and ready to do some real work. As it stands they are on a road to nowhere. We have been down that road and we know where it leads.
Posted in DECISION SUPPORT, EHR, HL7, MESSAGING, STANDARDS | No Comments »
February 29th, 2008 andrew
We know we want good semantics in messages and the best way to get that is atomic data in a standards based format. However the roadblock we run up against is that its not sexy enough!
The availability of computers rather than typewriters for Medical Notes means that the visual presentation of a clinic report or letter has become important to many and when you suggest that readers are really interested in the content rather than the presentation you often get that sinking feeling that you have just lost them.
So it seems to keep everyone happy we need both. However that sexy display needs to be portable and ideally standards based. It is also desirable to have some semantics in the display so a consumer of the report can eg. Click on a drug or diagnosis to drill down for more information. Ideally combining the atomic data in the HL7, which for example include images, with a standards based display format using xHTML and CSS, would fit the bill. Using Microformats it would also be possible to mark up diagnoses and drugs with there SNOMED-CT representation.
eg. Say you wanted to say the patient Had “Large B-Cell Lymphoma” – as Text this is not something you can reliably parse out, however if you used Microformats in xHTML you could represent it as:
<span title=”64572001|Disease| : 116676008 = 46732000|Malignant Lymphoma, large B-cell diffuse|”>Large B-Cell Lymphoma</span>
Or for usual more compact usage you could just use the SNOMED-CT codes alone:
<span title=”64572001 : 116676008 = 46732000″>Large B-Cell Lymphoma</span>
Now this embedded in the display text has real meaning and could be used as a link for disease information for example. The Australian HL7 V2 standard defines a html display segment, as the last OBX in a message and by transmitting the atomic data in the preceding OBX segments for machine usage and including a xHTML display segment you get the best of both worlds, atomic data and sexy display. If you add microformats to the xHTML then the display still has semantics that are safely extracted. The SNOMED-CT grammar can also go further and define such information as “This is a past history or Family History of a disease” which extends the semantics further.
Currently HL7 Freetext is used for display, and the only real formatting available is bold. This is causing some user pushback and a move to xhtml and CSS (probably with no javascript and inline CSS) would allow the embedding of good display into messages containing atomic data. HL7 CDA – Clinical Document Architecture allows to a html display segment as well so the concept is usable here as well. The alternatives would be rtf display, but xhtml is far more semantic and standard and is likely to be more inter-operable. There are free xhtml validators on the web and some form of view control is installed in virtually every OS available, which increases the attractiveness of xhtml.
It is an idea we are exploring at this time.
Posted in EHR, HL7, MESSAGING, SNOMED, STANDARDS, XHTML | No Comments »
February 26th, 2008 andrew
Analysis of SMTP
for Secure and Reliable Document Delivery.
By Peter R. Tattam
It has been proposed that secure email in the form of S/MIME over SMTP (Simple Mail Transfer Protocol) be used for the transmission of documents such as Electronic Medical Records (EMR) with high reliability. This article discusses the limitations of using SMTP for such delivery based on its poor suitability to meeting the requirements for high reliability document delivery, in particular in the context of that required for EMR transmission.
To begin discussion, one needs to define what the requirements for high reliability of document delivery might look like. There principal factors are as follows.
- Timely Delivery. One should expect that a document be delivered within an acceptable timeframe. This can vary depending on the needs of the document, however for many types of EMR, delivery times are expected to be in the order of seconds or minutes, especially when the nature of the data they convey is likely to age quickly.
- In-Order. One should expect that documents originating from the same source be delivered in the same order in which they originate from the source.
- No Duplicates. One should expect that an original document be received once and only once at the destination.
- Detection of delivery failure. One should be able to notify the sender that delivery failed for whatever reason.
- Secure Delivery. One should expect that a document be delivered intact and free of alteration and also be subject to typical expectations of a secure transmission. There are several aspects of this but the principal components are encryption and authentication.
- Safe from Unsolicited input. One should expect that documents be originated only by originators who are authorized to transmit such documents and that the system is free from unsolicited documents being injected into it.
- Safe from Hijacking. One should expect that the delivery system at both ends be predictable and free from interference.
This list is not necessarily exhaustive, but covers most of the needs required for reliable delivery.
Now let us see how SMTP deals with these factors. I base my analysis on the experience I have gathered in more than 15 years in the development of email clients and servers and from managing an ISP.
Timely Delivery.
The SMTP system can transmit an email through its system very quickly under ideal conditions. Typically, times are of the order of < 1 minute and comprise the sum of the following components.
1. Transmission by the MUA (Mail User Agent) to the first MTA (Mail Transfer Agent).
2. Transmission by Intermediary MTAs to other Intermediary MTAs or the Destination MTA. This step may occur zero or more times.
3. Receiving the document from the destination MTA to the destination MUA.
In step 1 and 3, there may be hidden processes that the user is unaware of, such as virus scanner software or corporate email conversion gateways when the internal mail protocols is not SMTP. Some of the steps involve complex and time variable operations such as looking up the DNS system which further compounds any delays.
There are many factors that can impede the throughput of the MTAs. These include server failures due to hardware failure, server overload due to SPAM (unsolicited email), email worms, and other hostile Denial-of-Service (DoS) attacks directed specifically at a given server. In the short term, email may be delayed anything from 30 minutes up to 4 hours (depending on site configuration) if an intermediary email server is not available for reasons such as network deterioration of any sort, or general unavailability due to maintenance etc. Generally users of email accept that delivery is not necessarily immediate, their experience being that of the real world “snail mail”, the physical postal system. A minor temporary delay caused by a communication failure between MTAs can often cost up to several hours even when the temporary delay lasts only for a brief time like a few seconds. There is no way for a MTA to determine when to resend an email based on network conditions although some servers do have some degree of intelligence if there are a large number of emails waiting to be sent to a particular MTA. Generally if an email can’t be sent, it will be queued for resending with a retry of between 30 mins to 4 hours. There are no rules for how often email should be resent – it is entirely arbitrary and at the whim of the administrators of the MTAs.
However, when a mail server is in a non-optimal state, such as when an internet worm is at the peak of its activity or a rather active group of spammers are busy, mail can be delayed for more than 24 hours, sometimes several days, especially if the source of unsolicited email into the system cannot be controlled. Clearly such a level of delay for important documents such as EMRs would be totally unacceptable, and generally when such events occur, they are totally outside the control of the sender or receiver of these documents.
Another contributory factor towards server delays is that very often in a large organization such as a Telco or large ISP, the mail server may be providing service for 100,000 users or more. Having that many user active, although not necessarily concurrently using the server, does place a high demand on general throughput, and often throughput is sacrificed for scalability.
In Order
There is no concept of ordering delivery of emails in SMTP. Mail will get delivered whenever it can. Mail can and will arrive out of order, especially if delays occur for the many reasons outlined above.
No Duplicates
Generally, the mail system will not send duplicates as usually the MTAs will take ownership of an email as it transitions through the system. There are however occasions when these systems fail in various ways, and it is possible to receive duplicate emails, for example if a MTA has suffered a catastrophic failure and needed to be restored from a backup, or a poorly configured server having been fixed. Failures and mistakes do occur in the administration of sometimes unwieldy MTAs. One other source of duplicates is that due to significant delays in transferring an email, a sender may resend the message.
Detection of Delivery Failure
The SMTP system by nature is a push system. There is no inbuilt mechanism to notify that the MUA of the recipient has received the document. Although there are modest extensions to SMTP for the acknowledgement of delivery, they generally rely on end-end acknowledgement of some kind in the MUA software.
Secure Delivery
This is a wide topic and encompasses the issues surrounding S/MIME. As with all security related issues, absolute best practice in the industry should be followed at all times. For truly secure transmissions to take place, both encryption and authentication with fully verifiable PKI must be followed. Any weak link will reduce the ultimate security. Proof of identity through authentication is absolutely necessary.
Safe from Unsolicited Input
SPAM, internet worms and viruses, and mail bombing can have a very adverse affect on both MTAs and MUAs. The essentially anonymous authentication system which the SMTP system is based around allows for unsolicited email to enter the system unchecked. While good configuration of servers is important to prevent SPAM, spammers are evolving more advanced methods to circumvent these measures. Also the proliferation of consumer based computing systems with less than average immunity to Viruses, Trojans and various other Malware has seen the rise of internet worms which can penetrate even the best engineered consumer ISP networks.
Safe from Hijacking
While technically a security related issue, there is a range of attacks at the network management layer which could interfere with the safety of email transmissions. When a MUA sends and receives emails, they generally do so via an internet account of some kind. For transmission of email, the MUAs use SMTP, the same protocol used for MTAs to communicate, however at the final receiving end, MUAs typically use POP3 or IMAP to receive the emails from the MTAs. This invariably requires an authenticated login of some kind. While there are measures to prevent eavesdropping of usernames and passwords in such a service, generally one is restricted by what the ISP supplies. Often there may be no choice of authentication method (plaintext passwords are common), and the ISP will often not allow the customer to operate their own SMTP server at the customer’s premises, nor will they allow direct access to SMTP servers other than those operated by the ISP. Since passwords are sent in plaintext over the network, it is entirely feasible that the account login may be hijacked, possibly without knowledge of the recipient. For example an eavesdropper may read mail without leaving a trace, and could also possibly remove emails without the genuine recipients knowledge. Even if the emails are secured with S/MIME, they could still use the knowledge to harvest email addresses for other purposes such as denial of service. Emails when sent and received carry a good deal of additional header information, and this information could be used by a hostile attacker to find weaknesses in the SMTP systems which comprise the virtual EHR system.
Other studies
There have been a number of studies made on the reliability of email for general use. A Google search on the phrase “email reliability” brings up a number of useful links. Here are some references to get started.
http://en.wikipedia.org/wiki/Email discusses the basics of email.
http://itmanagement.earthweb.com/columns/executive_tech/article.php/3500506 cites some studies.
http://www.broadbandreports.com/shownews/36176
http://www.emailwatchdog.org/
http://www.terryscomputertips.com/computers/email-reliability-in-this-internet-world.php
http://www.cbsnews.com/stories/2005/02/28/tech/main676962.shtml
Conclusion
This discussion only scratches the surface of the issues. Given the large number of points of failure for the SMTP system, it could hardly be relied upon for timely delivery of messages containing important documents where the age of the information is important. As for security, as long as best practices are followed, S/MIME should provide at least the same level of security as SSL or PGP encrypted HTTP since it uses tried and true Public Key Encryption. That however needs to be tempered with the reality that the login accounts of MUAs are still prone to hijacking in various forms. Since there are other more suitable protocols available that provide a more direct connection between endpoints (e.g. HTTP), one should consider that S/MIME be used only as a last resort when all else is unavailable. One would have a very poor internet experience indeed if the only services available were SMTP and POP3. That the SMTP system is the target of choice for the delivery of SPAM and other Malware should be sufficient warning to look at the alternatives.
My considered opinion is that using SMTP for the secure delivery of EMR is at best a poor second to better protocols like HTTP or SSH, and at worst a recipe for disaster - lawsuits from both the civil and medical fronts just waiting to happen.
Posted in EMAIL, HL7, MESSAGING, STANDARDS, Uncategorized | No Comments »
February 17th, 2008 andrew
It seems that after spending over $300M trying to kick start eHeath we are going to have an eHeath plan! Seems to me like that is a very good idea and the question is why don’t we have one already and what should be in “it”.
It’s an eHeath plan drafted in 3 months and will last 5-10 years! Still doesn’t sound quite right to me but any plan is better than no plan. Unless something major is changed its likely to be formulated by people with more political clout than technical expertise so maybe some ideas from the trenches would help.
Firstly, we don’t want a grand plan as they usually fail. We want to put in place measures that get us to a point that a grand plan is a possibility. A grand centralised EHR will not work unless everyone is using standards for their every day eHeath dabblings. That’s where the plan should focus. Australia has widespread usage of HL7 V2 and thats what we should aim for, but at a better level of quality. HL7 V3 has some nice modelling but is is not ready for widespread usage yet and the NHS in the UK is pouring buckets of money into that pond so we should swim in the functioning pool. CDA is not really any advance on good V2 and while the xml might be easier to parse it has no support in the current primary care applications and should not be an immediate target because of this.
What we don’t have is good quality HL7. The labs produce it but the receivers are fragile and the lowest common denominator is limiting what we can do. We need to have certification of the HL7 produced and certification of the applications ability to consume it. If this was done correctly then the reliability and quality of our clinical messaging would soar. We already have AHML able to certify HL7 produced and test message sets to test on applications so its easily organised! The State Heath Departments should be forced to only spit out 100% pure AHML accredited HL7, its not safe to do anything else and I am sure it could be done give enough push.
We do need provider directories, but almost all the lab messaging uses Medicare provider numbers for this and it works well. The grand all singing and dancing provider directory is a way off so why not use something that just works. Medicare Australia need to loosen their grip on the monthly lists of provider numbers for this to work but I am sure this could happen with only a modicum of counselling.
We need standards that allow you to send an entire patient history in HL7 V2 including SNOMED-CT encoded concepts. The archetypes in V2 project has the ability to allow this but this project has been running on a shoestring. Standards Australia needs to have its eHealth budget lifted by about 10 times. Currently there is one 1/2 time person for all the eHeath standards! Projects like this should have a full time person on them. It can take 2 months to get a document edited. This is not a Standards Australia problem, but a lack of funding support issue and it needs to be fixed ASAP. Someone, who can read and write HL7 in their sleep needs to be employed full time doing examples and testing concepts. Currently this is done by volunteers at midnight the night before the meeting.
We do also need messaging of course, but we need good quality messages before this is actually useful. The most useful option here would be to produce a standard and mandate its use. Even producing a standard would help as at the moment the “Use WS-Security” is nowhere near enough detail! How could this be done, this one would take some bravery but throwing together 10 technical (Not political) experts from the companies involved for a week with the expectation of a draft standard at the end of a week is probably the best option. Once that is done you could legislate that the receivers paid for the service if you were really brave. Next issue a directive to HESA (Now inside Medicare) to issue a site certificate and Individual certificate to everyone with a provider number and do it every time a new number is issued. You would also insist on a certificate signing service so certificates could be generated onsite, maybe using a valid individual key to upload it for signing.
Now drugs, we need SNOMED-CT codes and they need to be in the PBS by the end of the year. They also need to be integrated into the SNOMED-CT hierarchy so we can use them for decision support. A ministerial to advise that this must be done by the end of 2008 is what’s required here. SNOMED-CT may not be perfect but expecting clinicians to take responsibility for patients with no viable decision support is just not fair. When we have this we can look at scripts as they are just HL7 orders.
Next we need some money for all this, Standards Australia just need a one line budget, but perhaps we could add a”e” in front of every consultation/script that was messaged using AHML accredited HL7 and Medicare could pay the doctor/lab/pharmacy a premium and link all future rebate increases to this. That would certainly make it happen and you would only pay for performance. Its pointless paying vendors big $ to implement “X” – it has not worked. Reward some rubber on the road.
Ok, there it is, a 30 minute eHeath plan. All it does is put some onus on people to do what they should have already done and throw away the idea of expensive central systems and replace that with some good solid tested code in the trenches. That’s what we need.
Posted in DECISION SUPPORT, EHR, HL7, MESSAGING, SNOMED, STANDARDS, Uncategorized | No Comments »
February 17th, 2008 andrew
Our initial cross platform porting efforts were directed at OSX, but given its heritage we knew that linux would be close. It was quite easy in fact in the end and was running within a few hours. The yahoo user interface toolkit has been chosen for the UI and its certainly impressive. As capricorn is mainly a client for secure messaging it doesn’t not need much of a UI, but much could be done in a browser when combined with JSON and rest methods. Pictured is a form to send simple secure clinical messages directly from Firefox. Its useful to send quick notes but usually capricorn would integrate directly with other software.
Posted in HL7, LINUX, MESSAGING, SOAP, Uncategorized | No Comments »
February 2nd, 2008 andrew
This is the challenge of health IT at the primary care level. Most of the medical-objects destinations are Australian general practices which while computerised, are only just computerised and the stability and management of their systems is well… poor in general.
On top of this we run real time messaging using full PKI security for privacy and authentication. It is a challenge at times and we are continuing to learn, sometimes the hard way. The average life of a client install is < 12 months and often no one in the practice knows which computer is on, and frequently that computer disappears as part of an upgrade or reshuffle.
Managing this over thousands of sites is something we are trying to perfect. The first lesson we learnt was to get the content of the messages standards compliant. This has been vital as when we know that the payload meets Australian standards we can safely tweak the envelope to suite to recipients software, which often can’t handle compliant HL7. Compliant messages have allowed us to develop a large number of message modifiers or decorators which can be chained together in a huge number of ways to achieve a message that can be consumed. Even with a huge array of standard modifiers we still run into problems. Corporate systems are virtually impossible to tweak and the fact that they read to report date from the wrong place is something that you sometimes just have to accept. So the next element that makes it work is scriptable modifiers. We have a pascal script interface into our hl7 and xml parser that allow virtually any modification to be made. This is less routine and scalable that the canned modifiers, but is something you just need to have.
The other area that has proved vital is ongoing monitoring of server uptime. This allows us to detect clients with problems before someone notices that there have been no results for a while. Installation of new anti-virus software or changes to authentication parameters commonly cause this. We have also found sever instances of ISP misconfiguration of routers that block traffic between 2 clients.
Forwarding of errors in sending to support is another vital health monitor. Analysis of this is still not to the stage we would like it as many clients are disconnected after hours and weekends and the monitoring has to try and take this into account. Automating analysis of the problem is also an area we can improve on. We have started to identify patterns in the errors that mean something but the analysis is quite complex. We are considering using GELLO (Clinical decision support Language) to try and recognise these patterns in an automated way.
Double checking of new user and upgrade configuration is now much enhanced also. Users are good at finding shortcuts and at one stage we found support found a way to copy installation setups, followed by a few modifications. This made their setup much quicker but they did not change the GUID that is used for the URL on the proxy server…. The pleasing thing was that the PKI infrastructure prevented breaches of privacy as a wrong recipient could not decrypt the data but it had us scratching our heads. The sender and recipient both had each others keys and they were signed and correct but still we were getting errors in both authentication and decryption. In the end several clients were randomly getting each others messages so it would fail commonly, but if the correct client eventually received the message it would work. It was only when we noticed no hits in the client log despite a failed transmission that we tweaked to this one. The servers now avidly check all setups to prevent this happening.
The process has also made us aware of the difficulties in testing distributed systems. We have complex test setups but trying to simulate the real world of a distrubuted messaging system is also impossible. You have to try but the ongoing monitoring of connectivity and errors is something you have to have and the real world will always throw up surprises.
However we are excited about how well it works and how fast distributed systems are. The real world works quicker than any test setup and the message processing is done on different machines. In a test setup on a few machines you can almost follow it by looking at logging windows. In the real world it moves so fast that you have no hope of following it. Now to build something useful on top of it, after all that’s why we built it!
Posted in GELLO, HL7, MESSAGING, STANDARDS, Uncategorized | No Comments »
January 23rd, 2008 andrew
Its taken an enormous effort, but we are finally close to having the same code base compile on Windows, OSX and Linux.
The core HL7 code was not difficult but in the end we have implemented at lot of the threading and synchronisation calls that are in windows, but not in *nix. It have also meant implementing many complex routines in pure pascal. This has included a HTTP server, RSA/IDEA encryption and JSON and XML parsers! It has also driven us to the Web 2.0 world, which has been a pleasant drive!
Credit must be given to the amazing ability of Peter Tattam, who has been slaving away at this task on and off for the last few months on and off. His ability to work with low level code is second to none.
So where are we up to – we have a native pascal version of Capricorn, running and working well on OSX with the same source code base as the windows version. At this stage its not using any special libraries and is a single executable. We have dissected out or implemented all the windows specific code and are close to doing some beta testing. The scary bit is that in its current form its 300K lines of object pascal code… Luckily its mostly code that has been under extensive use in the Windows world for quite a while.
So here is a very raw low res sneak peak of Capricorn running as a Native OSX appication. It really isn’t much to look at as this is pure messaging infrastructure, but its important vital infrastructure if we are going to move into a world of real time standards based HL7 messaging!
Posted in EHR, HL7, MESSAGING | No Comments »