NNS Proposal for 64bit Stable Memory

Arthur Falls
Arthur Falls
8/30/2021

Summary: A proposal to increase IC memory addressing to 64bit, increasing canister storage from an effective 4GB to eventually the full capacity of a subnet: Currently 300GB.

Rationale: Applications need more than 4GB. A solution to enable applications to transparently shard state across multiple canisters, BigMap, is a heavyweight solution that potentially adds complexity. Increasing memory addressing is a simpler solution that serves the needs of most if not all builders on the IC today.

Concerns: Introduces potential failure modes for software written for 32bit addressing.

"The IC presents itself as having the ability to host code in perpetuity. The continued introduction of new failure modes makes this completely impossible." - Hazel Rowel

Proposal:
Proposal verbiage is in-depth, concise, and comprehensible. It accurately represents the logic content of the proposal.

Discussion: The 4GB limit on canister storage was always intended to be temporary and a solution needs to be adopted that is developer friendly, backward compatible, complete, and doesn't introduce unnecessary complexity.

As most applications will be able to fit inside 300GB, increasing canister capacity will have the greatest benefit for the greatest number of developers. BigMap will still be necessary for storing larger volumes of data. However, by increasing the canister capacity, the urgency is removed from BigMap, and fewer applications will need to use it. 

However, as identified by the proposal documentation, there are potential risks of this upgrade, primarily relating to how it is used by developers. 

An issue raised by DFINITY member akhilesh.singhania on the DFINITY Forums discussed the future development and upgradability of the API represented by this proposal and future APIs. cycle_dao would like to see a clearly defined process for the rollout of experimental features, including indications of developer best practices.

Ex-DFINITY engineer Nomeata referenced an unpublished document “Refining persistence” by Andreas Rossberg outlining the vision for memory management.

Conclusion:

A positive differentiator of the IC from other blockchain computers is an ethos of upgradability. A negative consequence of that differentiator is platform instability. A changing platform without concrete dedication to backward compatibility has the potential to undermine the vision of unstoppable software. As much as possible the introduction of risks to currently running software should be avoided and the stability of the execution environment should be preserved. The proposal discussion clearly demonstrates these concerns are foremost in the minds of DFINITY engineers. The proposal is a well considered solution to the issue of limited canister storage capacity and cycle_dao supports the approach.

We would like to see a forward-looking, living document that identifies possible future proposals that may affect backward-compatibility. This document should define best practices for developing unstoppable software today that will not be affected by future proposals.

cycle_dao will vote Yes on this proposal.


More Stories

NNS Proposal: Enabling Canisters to hold ICP

9/16/2021

Enabling canisters to hold ICP - currently only user accounts and the NNS can hold ICP

Arthur Falls
Arthur Falls

NNS Proposal: Direct Integration with Bitcoin

9/15/2021

A proposal to enable users and canisters on the Internet Computer to transact wrapped bitcoin directly without the use of a centralised intermediary or an intermediary protocol. This is achieved by IC replicas running Bitcoin nodes. 

Arthur Falls
Arthur Falls