Finished Reviews

PQ Review Process Differences

Description of changes for Process Evolution

Versions

0.6.1 December 2020

Added % guidance for Report of the Results in the Testing section

0.6 -- 6 November 2020

Overview

  1. Improved “Summary of the Process”

  2. Improved Disclaimer after legal review

  3. Improved “Sections of the Review Process” to match the revised report

  4. Changed the Scoring rubric to match all 0.6 changes

  5. Added “Private Software Repos” section

Code and Team

  1. Improve the wording in the “Are the executing code addresses readily available? (Y/N)” question to emphasize its importance and the impact of not having the addresses public.

  2. Removed the question; "Are the Contracts Verified?". This question asks if the contracts are "verified" on Etherscan. Frankly, it was always true on every review we did. For this reason, I am removing the question as it adds no value.

  3. Changed the question " Does the code match a tagged version on a code hosting platform" to "Is There a Public Software Repository". Finding and matching all the contracts was time-consuming and added little value.

  4. Change the question from "Is the Software Repository Healthy" to "Is There Development History Visible?". This is effectively the same question asked in a different way. When asked this way the software repository is not mandatory, merely convenient.

  5. Added the question “Is the team public (not anonymous)? (Y/N)” as this is an element of trust we felt needed to be added.

Documentation

  1. Improved the wording in the "Is there a white paper?" question allowing medium articles also

  2. Change the wording of the question from "Do the requirements fully (100%) cover the deployed contracts?” to “Does the software function documentation fully (100%) cover the deployed contracts?” because many people do not really understand the word "requirements". Also added guidance to this question

  3. Added guidance based on Comments to Code (CtC) ratio on the question “Are there sufficiently detailed comments for all functions within the deployed contract code?” Change the wording of the question from “Is it possible to trace software requirements to the implementation in code” to “Is it possible to trace from software documentation to the implementation in code” because many people do not really understand the word "requirements". Test

  4. Added guidance based on Test to Code ratio (TtC) to the “Full test suite” is a good

Audits

  1. Added words where audits do not cover economic issues and the specific impact of audits on private repos

0.5 -- 7 August 2020

Added % Guidance to "Does the code match a tagged version on a code hosting platform (GitHub, GitLab, etc.)?"

Added % Guidelines to "Code coverage (Covers all the deployed lines of code, or explains misses) (%)" in Testing

Added % Guidelines to "Is it possible to trace requirements to the implementation in code (%)"

0.4 -- 24 July 2020

General rebalancing of the scoring weights

Added Summary of the Process section

Changed Deployed code to "Executing Code Verification"

Changed Requirements to Documentation

Scoring weight for the "deployed code address(s) readily available?" from 10 to 30 because it is fundamentally important

Scoring weight for the "Does the code match a tagged version on a code hosting platform (GitHub, GitLab, etc.)?" from 10 to 20

Scoring weight for the "Is development software repository healthy)?" from 20 to 10

Changed the heading of Requirements to Documentation for better clarity for the reader.

Deleted "Are the requirements available publicly? Question as it added little value.

Scoring weight for the "Are there sufficiently detailed comments for all functions within the deployed contract code?" from 5 to 10 because is important

Scoring weight for the "Code coverage", "Scripts and instructions to run the tests" and "Packaged with the deployed code" from 10 to 5 for balancing

Changed the weight of Audit from 50 to 70 for balancing

0.3 -- June 2020

"Is development software repository healthy?" of "Deployed Code" changed from Y/N to %. The AAVE code was developed in a private repository that the auditor cannot view. The public repository was created just to display the final code. This makes the public repository appear unhealthy. But as they have a valid reason and the auditor is comfortable a valid repository exists but cannot be seen we needed something better than a binary Y/N. So we changed to % and changed the explanation.

0.2 -- June 2020

This is the initial version used for the first three Audits