Reference data is often overlooked in the rush to get coding. In reality, much of ....
<h3>III. Historical Considerations</h3>
Before the existence of acs-reference, the ACS required that you preload some tables in a script to get some basic reference functionality. There were many problems with this:
Before the existence of acs-reference, the ACS required that you preload some tables in a script to get some basic reference functionality. There were many problems with this:
</p>
<ul>
<ul>
<li>No easy way to find out what reference data even existed.</li>
<li>No easy way to find out what reference data even existed.</li>
<li>No way to find out how old the data was.</li>
<li>No way to find out how old the data was.</li>
<li>No way to find out where that data came from.</li>
<li>No way to find out where that data came from.</li>
<li>Very US/English slant on the data.</li>
<li>Very US/English slant on the data.</li>
</ul>
</ul>
<h3> IV. Competitive Analysis</h3>
<h3>III. Design Tradeoffs</h3>
The only real competition is internally developed solutions.
<h3> V. Design Tradeoffs</h3>
<h4>Primary Goals</h4>
<h4>Primary Goals</h4>
<ul>
<ul>
<li>This system was designed with maintainability and reusability as its primary goals. By wrapping a layer around all of the reference tables we have increased the maintainability immensely.</li>
<li>This system was designed with maintainability and reusability as its primary goals. By wrapping a layer around all of the reference tables we have increased the maintainability immensely.</li>
...
@@ -35,26 +30,44 @@ The only real competition is internally developed solutions.
...
@@ -35,26 +30,44 @@ The only real competition is internally developed solutions.
<h4>Performance</h4>
<h4>Performance</h4>
When updating the reference tables their is overhead due to the fact that the table is registered with the repository. This should rarely occur anyway as the tables are only added once.
When updating the reference tables their is overhead due to the fact that the table is registered with the repository. This should rarely occur anyway as the tables are only added once.
By not having the actual data itself in the acs-object system, subsequent additions and deletions to the reference tables themselves are unaffected by this overhead.
By not having the actual data itself in the acs-object system, subsequent additions and deletions to the reference tables themselves are unaffected by this overhead.
<p>The UNSPSC reference data has a data model for handling data
There will
revisions. An application can determine any new/revised category based
<h3> IX. Configuration/Parameters</h3>
on existing, obsolete data.</p>
None
<h3>VI. User Interface</h3>
<h3> X. Future Improvements/Areas of Likely Change</h3>
<p>Their is no end user interface.
A server based update mechanism will be supported. This will allow for tables to be updated (and preferably diffed) instead of being reloaded with a package upgrade.
There needs to be some kind of admin UI to report status and possibly
manage updates per requirements.
</p>
<h3>VII. Configuration/Parameters</h3>
<p>None</p>
<h3>VIII. Future Improvements/Areas of Likely Change</h3>
<p>A server based update mechanism will be supported. This will allow for tables to be updated (and preferably diffed) instead of being reloaded with a package upgrade.
An interface to produce xml/csv from the reference data would be a nice service to the community (allowing legacy applications a way to import this data).
An interface to produce xml/csv from the reference data would be a nice service to the community (allowing legacy applications a way to import this data).
<h3> XI. Authors</h3>
</p>
<h3> XII. Revision History</h3>
<h3>IX. Authors</h3>
<p>
Jon Griffin
</p>
<h3>X. Pre-CVS Revision History</h3>
<pre>
<pre>
$Log$
$Log$
Revision 1.4 2006/08/06 20:40:20 torbenb
upgrading html, closing li p tags, adding quotes to tag attributes
This document describes the requirements for the ACS Reference service
<p>This document describes the requirements for the ACS Reference service
package. This package has the following primary functions:
package. This package has the following primary functions:
</p>
<ul>
<ul>
<li>It allows applications to refer to and employ a common set of reference
<li>It allows applications to refer to and employ a common set of reference
data.</li>
data.</li>
...
@@ -22,7 +24,9 @@ data of this sort.</li>
...
@@ -22,7 +24,9 @@ data of this sort.</li>
very often and also in many cases comes from an external source and not
very often and also in many cases comes from an external source and not
from within the system itself. Many times it is created from a standards
from within the system itself. Many times it is created from a standards
body, i.e. <ahref="http://www.iso.ch/">ISO</a> or <ahref="http://www.ansi.org">ANSI</a>, and may be required for a client's particular industrial needs.
body, i.e. <ahref="http://www.iso.ch/">ISO</a> or <ahref="http://www.ansi.org">ANSI</a>, and may be required for a client's particular industrial needs.
</p>
<p>Some examples of reference data are:
<p>Some examples of reference data are:
</p>
<ul>
<ul>
<li>Geographic data: zip codes, country codes and states/provinces</li>
<li>Geographic data: zip codes, country codes and states/provinces</li>
<li>Standards bodies data: ISO 4217 currency codes, ISO 3166 Country Codes, ITU
<li>Standards bodies data: ISO 4217 currency codes, ISO 3166 Country Codes, ITU
<li>Internal: Status Codes, Employee Position Codes</li>
<li>Internal: Status Codes, Employee Position Codes</li>
</ul>
</ul>
Historically, reference data has been looked upon by developers as
<p>Historically, reference data has been looked upon by developers as
something less important than more immediate coding needs, and so most
something less important than more immediate coding needs, and so most
data models simply defer the issue by treating reference data as
data models simply defer the issue by treating reference data as
something simple to implement. Elsewhere. The reality is
something simple to implement. Elsewhere. The reality is
that for most organizations reference data is extremely important and
that for most organizations reference data is extremely important and
also extremely difficult to manage.
also extremely difficult to manage.
</p>
<p>This module will not only <i>package</i> all of a site's reference
<p>This module will not only <i>package</i> all of a site's reference
data in one place, it will also help manage that data.
data in one place, it will also help manage that data.
</p>
<h3>III. System Overview</h3>
<h3>III. System Overview</h3>
The ACS Reference package consists of:
<p>The ACS Reference package consists of:</p>
<ul>
<ul>
<li>A standard framework for monjitoring and modifying reference data.
<li>A standard framework for monjitoring and modifying reference data.</li>
<li>A method of determining whether or not that data is expired.
<li>A method of determining whether or not that data is expired.</li>
<li>The ability to include not only the data but also functions to
<li>The ability to include not only the data but also functions to
work with that data.
work with that data.</li>
</ul>
</ul>
<h3>IV. Use-cases and User-Scenarios</h3>
<h3>IV. Use-cases and User-Scenarios</h3>
Papi Programmer is developing a module that will use country codes as
<p>Papi Programmer is developing a module that will use country codes as
part of his table structure. Instead of creating his own table he can
part of his table structure. Instead of creating his own table he can
use the ACS Reference package and the country codes therein. If the
use the ACS Reference package and the country codes therein. If the
country codes change - which does in fact happen from time to time -
country codes change - which does in fact happen from time to time -
the ACS Reference package will maintain that information for him.
the ACS Reference package will maintain that information for him.
</p>
<h3>V. Related Links</h3>
<h3>V. Related Links</h3>
<ul>
<ul>
<li><ahref=design.html>Design document</a>
<li><ahref="design.html">Design document</a>
</ul>
</li></ul>
<h3>VI.A Requirements: Data Model</h3>
<h3>VI.A Requirements: Data Model</h3>
<p>
10.10 The package should use a table that is the <i>master</i> table for all reference tables.
10.10 The package should use a table that is the <i>master</i> table for all reference tables.
<br>10.20 The package should employ a field to show whether this data is internally derived or not.
<br>10.20 The package should employ a field to show whether this data is internally derived or not.
<br>10.30 The package should employ a field to signify whether there is a PL/SQL package involved with
<br>10.30 The package should employ a field to signify whether there is a PL/SQL package involved with
...
@@ -69,40 +76,51 @@ this table.
...
@@ -69,40 +76,51 @@ this table.
<br>10.70 The package should offer a representation of effective datetime
<br>10.70 The package should offer a representation of effective datetime
<br>10.80 The package should offer a representation of discontinued datetime
<br>10.80 The package should offer a representation of discontinued datetime
<br>10.90 The package should keep an indication of who the data maintainer is, by user_id.
<br>10.90 The package should keep an indication of who the data maintainer is, by user_id.
</p>
<h3>VI.B Requirements: API</h3>
<h3>VI.B Requirements: API</h3>
<p>
20.10 The package should offer a function to determine if a particular table has expired.<p>
20.10 The package should offer a function to determine if a particular
table has expired.
The requirements below are not met by the current implementation:<p>
</p><p>
The requirements below are not met by the current implementation:
</p><p>
30.10 There needs to be a way to query the data source and update
30.10 There needs to be a way to query the data source and update
automatically. If that isn't possible, as it won't be in many cases,
automatically. If that isn't possible, as it won't be in many cases,
the application should be able to query a master server and see if
the application should be able to query a master server and see if
there is new data for a particular table or tables. For example:
there is new data for a particular table or tables. For example:
refdata.arsdigita.com could hold the reference tables and when newer
refdata.arsdigita.com could hold the reference tables and when newer
table versions become available, simply upload only these versions or
table versions become available, simply upload only these versions or
perhaps even only the differences between the tables.
perhaps even only the differences between the tables. In any case,
there should be an admin page that shows current status and revisions
of various data, where to find info about additional sources (if
applicable), and provide a UI to upload or import new data.
</p>
<h3>VII. Implementation Notes</h3>
<h3>VII. Implementation Notes</h3>
<p>
The package needs to handle changes to reference data in a graceful
The package needs to handle changes to reference data in a graceful
fashion. For example, if a country splits into two or more countries, what should happen?
fashion. For example, if a country splits into two or more countries, what should happen?
</p>
<ul>
<ul>
<li>The reference package should note this change.</li>
<li>The reference package should note this change.</li>
<li>The appropriate table is updated. In this case countries et al.</li>
<li>The appropriate table is updated. In this case countries et al.</li>
<li>An update to the repository database field effective_date is added.</li>
<li>An update to the repository database field effective_date is added.</li>
<li>A <i>diff</i> type of entry into the reference repository history. <fontcolor = "red"><i>This is not in the current data model</i></font>
<li>A <i>diff</i> type of entry into the reference repository history. <fontcolor = "red"><i>This is not in the current data model</i></font></li>
<li>Then any sub-programs using this data will note the change of effective date and be able to handle the change as needed (i.e. simply read the new table).</li>
<li>Then any sub-programs using this data will note the change of effective date and be able to handle the change as needed (i.e. simply read the new table).</li>
<li>Historical data will be available using this <i>diff</i> for those applications that need to use the old data</li>
<li>Historical data will be available using this <i>diff</i> for those applications that need to use the old data</li>
</ul>
</ul>
Note also that it is possible to have overlapping effective dates.
<p>Note also that it is possible to have overlapping effective dates.
This will not be implemented in the first version, but should be recognized and accomodated throughout the development
This will not be implemented in the first version, but should be recognized and accomodated throughout the development
process for the service package.
process for the service package.
</p>
<h3> VIII. Revision History</h3>
<h3> VIII. Pre-CVS Revision History</h3>
<pre>
<pre>
$Log$
$Log$
Revision 1.3 2006/08/06 20:40:20 torbenb
upgrading html, closing li p tags, adding quotes to tag attributes
Revision 1.2 2006/08/06 18:41:43 torbenb
removed c-Ms, added p tags, added a comment to unimplemented requirements / feature request
Revision 1.1 2001/04/22 00:53:12 jong
Revision 1.1 2001/04/22 00:53:12 jong
initial openacs import
initial openacs import
...
@@ -123,10 +141,7 @@ reorganized content, edited for clarity
...
@@ -123,10 +141,7 @@ reorganized content, edited for clarity
