Maker DAO Process Quality Review

This is a MakerDAO Process Quality Audit completed on August 2020. It was performed using the Process Audit process (version 0.5) and is documented here. The audit was performed by ShinkaRex of Caliburn Consulting. Check out our Telegram.

This is NOT an audit of Maker Oasis which will have a separate audit.

The final score of the audit is 85%, a strong pass. The breakdown of the scoring is in Scoring Appendix.

Summary of the Process

Very simply, the audit looks for the following declarations from the developer's site. With these declarations, it is reasonable to trust the smart contracts.

  1. Here is my smart contract on the blockchain

  2. You can see it matches a software repository used to develop the code

  3. Here is the documentation that explains what my smart contract does

  4. Here are the tests I ran to verify my smart contract

  5. Here are the audit(s) performed to review my code by third party experts

Disclaimer

This report is for informational purposes only and does not constitute investment advice of any kind, nor does it constitute an offer to provide investment advisory or other services. Nothing in this report shall be considered a solicitation or offer to buy or sell any security, future, option or other financial instrument or to offer or provide any investment advice or service to any person in any jurisdiction. Nothing contained in this report constitutes investment advice or offers any opinion with respect to the suitability of any security, and the views expressed in this report should not be taken as advice to buy, sell or hold any security. The information in this report should not be relied upon for the purpose of investing. In preparing the information contained in this report, we have not taken into account the investment needs, objectives and financial circumstances of any particular investor. This information has no regard to the specific investment objectives, financial situation and particular needs of any specific recipient of this information and investments discussed may not be suitable for all investors.

Any views expressed in this report by us were prepared based upon the information available to us at the time such views were written. Changed or additional information could cause such views to change. All information is subject to possible correction. Information may quickly become unreliable for various reasons, including changes in market conditions or economic circumstances.

Executing Code Verification

This section looks at the code deployed on the Mainnet that gets audited and its corresponding software repository. The document explaining these questions is here. This audit will answer the questions;

  1. Is the executing code address(s) readily available? (Y/N)

  2. Is the code actively being used? (%)

  3. Are the Contract(s) Verified/Verifiable? (Y/N)

  4. Does the code match a tagged version in the code hosting platform? (%)

  5. Is the software repository healthy? (%)

Is the executing code address(s) readily available? (Y/N)

Answer: Yes

You have to dig a bit to find the deployed contract addresses. First pick Developers from the website, then Documentation. From there when you dig to a particular module, with the detailed documentation they have the Etherscan address. A bit different, but effective.

They are available at Address 0x6B175474E89094C44Da98b954EedeAC495271d0F as indicated in the Appendix. This Audit only covers the "dai" contract deployed on Nov 13, 2019.

Is the code actively being used? (%)

Answer: 100%

Activity is in excess of 2,000 transactions a day, as indicated in the Appendix.

Percentage Score Guidance

100% More than 10 transactions a day

70% More than 10 transactions a week

40% More than 10 transactions a month

10% Less than 10 transactions a month

0% No activity

Are the Contract(s) Verified/Verifiable? (Y/N)

Answer: Yes

0x6B175474E89094C44Da98b954EedeAC495271d0F is the Etherscan verified contract address.

Does the code match a tagged version on a code hosting platform? (%)

Answer: 30%

The code for dai.sol did match perfectly but I had to go find the code manually based on the date of the deployed code. There was no release or clear annunciation of the deployed code. When I check cat.sol, I found some small additions to the code not in the repository so finally I could not fully matching code.

As per guidance below 30%

Guidance:

100% Code matches and Repository was clearly labelled 60 % Code matches but no labelled repository. Repository was found manually 30% Code does match perfectly and repository was found manually 0% Matching Code could not be found

GitHub address : https://raw.githubusercontent.com/makerdao/dss/6fa55812a5fcfcfa325ad4d9a4d0ca4033c38cab/src/dai.sol

Deployed contracts in the following file;

Matching Repository: https://github.com/makerdao/dss/tree/6fa55812a5fcfcfa325ad4d9a4d0ca4033c38cab

How to improve this score

Ensure there is a clearly labelled repository holding all the contracts, documentation and tests for the deployed code. Ensure an appropriately labeled tag exists corresponding to deployment dates. Release tags are clearly communicated.

Is development software repository healthy? (%)

The repository has 373 commits and 33 branches. Healthy as per the Appendix.

Answer: 100%

Documentation

This section looks at the software documentation. The document explaining these questions is here.

Required questions are;

  1. Is there a whitepaper? (Y/N)

  2. Are the basic application requirements documented? (Y/N)

  3. Do the requirements fully (100%) cover the deployed contracts? (%)

  4. Are there sufficiently detailed comments for all functions within the deployed contract code (%)

  5. Is it possible to trace software requirements to the implementation in code (%)

Is there a whitepaper? (Y/N)

Answer: Yes

Location: https://makerdao.com/en/whitepaper/

Are the basic application requirements documented? (Y/N)

Answer: Yes

Location: https://docs.makerdao.com/smart-contract-modules/core-module

Do the requirements fully (100%) cover the deployed contracts? (%)

Answer: 100%

Maker has an excellent, very complete documentation set that appears to fully cover the code.

Are there sufficiently detailed comments for all functions within the deployed contract code (%)

Answer: 45%

As per the code examples there is only minimal commenting in the actual code. Despite excellent top level documentation, there is little connection and the code alone does not explain or connect with the documentation.

Code examples are in the Appendix. As per the SLOC, there is 21% commenting to code.

How to improve this score

This score can improve by adding comments to the deployed code such that it comprehensively covers the code. For guidance, refer to the SecurEth Software Requirements. If good top level documentation exists clearly connect the code to the documentation so the reader is easily able to find the code that covers the explained text.

Is it possible to trace requirements to the implementation in code (%)

Answer: 40%

While the names of most functions are listed in the documentation with a

Guidance: 100% - Clear explicit traceability between code and documentation at a requirement level for all code 60% - Clear association between code and documents via non explicit traceability 40% - Documentation lists all the functions and describes their functions 0% - No connection between documentation and code

How to improve this score

This score can improve by adding traceability from requirements to code such that it is clear where each requirement is coded. For reference, check the SecurEth guidelines on traceability.

Testing

This section looks at the software testing available. It is explained in this document. This section answers the following questions;

  1. Full test suite (Covers all the deployed code) (%)

  2. Code coverage (Covers all the deployed lines of code, or explains misses) (%)

  3. Scripts and instructions to run the tests (Y/N)

  4. Packaged with the deployed code (Y/N)

  5. Report of the results (%)

  6. Formal Verification test done (%)

  7. Stress Testing environment (%)

Is there a Full test suite? (%)

Answer: 100%

There is a separate test file for almost each solidity file. The test files stay with each branch of the source files. There is a 171% of tests to code as per the SLOC data.

Code coverage (Covers all the deployed lines of code, or explains misses) (%)

Answer: 50%

There is no evidence of organized collection of code coverage. However with the comprehensive set of tests, some coverage would be found. 50% as per guidance below.

Guidance: 100% - Documented full coverage 99-51% - Value of test coverage from documented results 50% - No indication of code coverage but clearly there is a reasonably complete set of tests 30% - Some tests evident but not complete 0% - No test for coverage seen

How to improve this score

This score can improve by adding tests achieving full code coverage. A clear report and scripts in the software repository will guarantee a high score.

Scripts and instructions to run the tests (Y/N)

Answer: Yes

Yes in travis.yml.

Packaged with the deployed code (Y/N)

Answer: Yes

The code is in a separate test directory off of the src directory.

Report of the results (%)

Answer: 0%

No test result report or coverage report was visible. Not in the deployed repository or the master/dss.

Formal Verification test done (%)

Answer: 100%

A report on the Formal Verification done: https://security.makerdao.com/formal-verification

Stress Testing environment (%)

Answer: 100%

A report on the Runtime verification done: https://forum.makerdao.com/t/publication-of-the-runtime-verification-audit/976

Audits

Answer: 100%

Two detailed and comprehensive audits took place on the MakerDao contracts before deployment. Some vulnerabilities were found. We assume these were fixed, but actual evidence of the fixes were not seen. Overall an excellent securities effort.

Trail of Bits Audit: https://github.com/makerdao/mcd-security/blob/master/Audit%20Reports/TOB_MakerDAO_Final_Report.pdf

Peckshield Audit: https://github.com/makerdao/mcd-security/blob/master/Audit%20Reports/PeckShield_Final_Audit_Report.pdf

  1. Multiple Audits performed before deployment and results public and implemented or not required (100%)

  2. Single audit performed before deployment and results public and implemented or not required (90%)

  3. Audit(s) performed after deployment and no changes required. Audit report is public. (70%)

  4. No audit performed (20%)

  5. Audit Performed after deployment, existence is public, report is not public and no improvements deployed (0%)

Appendices

Author Details

The author of this audit is Rex of Caliburn Consulting.

Email : rex@caliburnc.com Twitter : @ShinkaRex

I started with Ethereum just before the DAO and that was a wonderful education. It showed the importance of code quality. The second Parity hack also showed the importance of good process. Here my aviation background offers some value. Aerospace knows how to make reliable code using quality processes.

I was coaxed to go to EthDenver 2017 and there I started SecuEth.org with Bryant and Roman. We created guidelines on good processes for blockchain code development. We got EthFoundation funding to assist in their development.

Process Quality Audits are an extension of the SecurEth guidelines that will further increase the quality processes in Solidity and Vyper development.

Career wise I am a business development for an avionics supplier.

Scoring Appendix

Executing Code Appendix

Code Used Appendix

Repository Healthy Appendix

Example Code Appendix

contract Vat {
// --- Auth ---
mapping (address => uint) public wards;
function rely(address usr) external note auth { require(live == 1, "Vat/not-live"); wards[usr] = 1; }
function deny(address usr) external note auth { require(live == 1, "Vat/not-live"); wards[usr] = 0; }
modifier auth {
require(wards[msg.sender] == 1, "Vat/not-authorized");
_;
}
mapping(address => mapping (address => uint)) public can;
function hope(address usr) external note { can[msg.sender][usr] = 1; }
function nope(address usr) external note { can[msg.sender][usr] = 0; }
function wish(address bit, address usr) internal view returns (bool) {
return either(bit == usr, can[bit][usr] == 1);
}
// --- Data ---
struct Ilk {
uint256 Art; // Total Normalised Debt [wad]
uint256 rate; // Accumulated Rates [ray]
uint256 spot; // Price with Safety Margin [ray]
uint256 line; // Debt Ceiling [rad]
uint256 dust; // Urn Debt Floor [rad]
}
struct Urn {
uint256 ink; // Locked Collateral [wad]
uint256 art; // Normalised Debt [wad]
}
mapping (bytes32 => Ilk) public ilks;
mapping (bytes32 => mapping (address => Urn )) public urns;
mapping (bytes32 => mapping (address => uint)) public gem; // [wad]
mapping (address => uint256) public dai; // [rad]
mapping (address => uint256) public sin; // [rad]
uint256 public debt; // Total Dai Issued [rad]
uint256 public vice; // Total Unbacked Dai [rad]
uint256 public Line; // Total Debt Ceiling [rad]
uint256 public live; // Access Flag
// --- Logs ---
event LogNote(
bytes4 indexed sig,
bytes32 indexed arg1,
bytes32 indexed arg2,
bytes32 indexed arg3,
bytes data
) anonymous;
modifier note {
_;
assembly {
// log an 'anonymous' event with a constant 6 words of calldata
// and four indexed topics: the selector and the first three args
let mark := msize // end of memory ensures zero
mstore(0x40, add(mark, 288)) // update free memory pointer
mstore(mark, 0x20) // bytes type data offset
mstore(add(mark, 0x20), 224) // bytes size (padded)
calldatacopy(add(mark, 0x40), 0, 224) // bytes payload
log4(mark, 288, // calldata
shl(224, shr(224, calldataload(0))), // msg.sig
calldataload(4), // arg1
calldataload(36), // arg2
calldataload(68) // arg3
)
}
}
// --- Init ---
constructor() public {
wards[msg.sender] = 1;
live = 1;
}
// --- Math ---
function add(uint x, int y) internal pure returns (uint z) {
z = x + uint(y);
require(y >= 0 || z <= x);
require(y <= 0 || z >= x);
}
function sub(uint x, int y) internal pure returns (uint z) {
z = x - uint(y);
require(y <= 0 || z <= x);
require(y >= 0 || z >= x);
}
function mul(uint x, int y) internal pure returns (int z) {
z = int(x) * y;
require(int(x) >= 0);
require(y == 0 || z / y == int(x));
}
function add(uint x, uint y) internal pure returns (uint z) {
require((z = x + y) >= x);
}
function sub(uint x, uint y) internal pure returns (uint z) {
require((z = x - y) <= x);
}
function mul(uint x, uint y) internal pure returns (uint z) {
require(y == 0 || (z = x * y) / y == x);
}
// --- Administration ---
function init(bytes32 ilk) external note auth {
require(ilks[ilk].rate == 0, "Vat/ilk-already-init");
ilks[ilk].rate = 10 ** 27;
}
function file(bytes32 what, uint data) external note auth {
require(live == 1, "Vat/not-live");
if (what == "Line") Line = data;
else revert("Vat/file-unrecognized-param");
}
function file(bytes32 ilk, bytes32 what, uint data) external note auth {
require(live == 1, "Vat/not-live");
if (what == "spot") ilks[ilk].spot = data;
else if (what == "line") ilks[ilk].line = data;
else if (what == "dust") ilks[ilk].dust = data;
else revert("Vat/file-unrecognized-param");
}
function cage() external note auth {
live = 0;
}

SLOC Appendix

Solidity Contracts

Language

Files

Lines

Blanks

Comments

Code

Complexity

Solidity

13

2155

255

463

1437

301

Comments to Code 463/ 2155 = 21%

Javascript Tests

Language

Files

Lines

Blanks

Comments

Code

Complexity

Solidity

10

3163

460

245

2458

15

Tests to Code 2458/ 1437= 171%