|
Attachment
to the Joint Statement
WHAT IS A PROPER SOURCE CODE REVIEW?
By
AES Watch
Feb. 22, 2010
The
review of the source code of the Automated Election System by interested
political parties and groups is a right enshrined in RA9369. A proper
source code review is one that is conducted comprehensively following
IT industry best practices and accepted methodologies in software
quality and integrity assurance and using automated tools for testing
parts or the whole of the code.
Purpose
of the review: The purpose of the review is to check for
conformity of the AES 2010 computer programs (EMS, PCOS, CCS, and
related utilities) to the provisions of RA9369 and related Philippine
election laws, the COMELEC Terms of Reference to Bidders 2009, the
COMELEC General Instructions and related COMELEC implementing rules
and regulations, and for conformity to US EAC 2005 VVSG, wherever
this US standard does not conflict with Philippine elections laws
and regulations.
Basis
for the review: The review must be based on a set of standards.
In the case of the AES, we propose the use of the EAC VVSG 2005,
the Voluntary Voting System Guidelines adopted by the Election Assistance
Commission of the United States in December 2005. The purpose of
the VVSG is “to provide a set of specifications and requirements
against which voting systems can be tested to determine if the systems
provide all of the basic functionality, accessibility and security
capabilities required of voting systems.” The same VVSG was
presumably used by SysTest Labs for the review and certification
it performed.
Coverage
of the Review: The review must cover the source codes of
all the programs running the various components of the AES, including
but not limited to the source codes of the Election Management System
(EMS), the Consolidation and Canvassing System (CCS), Election Event
Designer and Precinct Count Optical Scanner (PCOS), Election Programming
Station (EPS), and other components, including the operating system
in the PCOS (customized cLinux as disclosed in the Smartmatic’s
SAES product description), the signing software, and transmission
software, and the encryption software.
Format
of the Source Code: The source code must be made available
in editable format, which may be used with document or programming
editing tools and/or automated software testing tools. COMELEC must
provide a copy of the baseline source code and of the customized
source code of all the AES software components.
Required
Documentation : The reviewers must be provided all existing
documentation related to the AES, including comments embedded within
the source code, preferably comments in each module explaining what
the particular module does. The documents that must be supplied
by the Comelec and Smartmatic-TIM include but are not limited to:
(a) Architectural Overview of the software, (b) Code Overview, how
it is structured, its logic and interfaces, (c) Functional Summary.
The user and administration manuals of the AES and its components
must also be provided. The Comelec must also provide the specifications
document that led to the customization of the AES. A more exhaustive
list of required documentation may be found in EAC VVSG 2005. In
addition, the SysTest Labs report must be shared with the reviewers,
as well as its test procedures and test results.
Test
Facility and Environment: The review may be conducted in
a controlled facility designated by Comelec but it must be a comfortable
area with ample workspace large enough to accommodate at least ten
reviewers at any one time and open beyond normal office hours each
day. Comelec must also provide the reviewers with a sufficient number
of computer workstations and/or notebooks with printers as well
as the software of automated tools needed for the review.
|
