Skip to content

README-DEV.md

Latest commit

 

History

History
264 lines (193 loc) · 12.2 KB

README-DEV.md

File metadata and controls

264 lines (193 loc) · 12.2 KB
Raw

Development Guideline

Please note, that the only officially supported platform now is Linux. It's recommended to develop and deploy the App on Ubuntu 24.04. It also works on other versions of Ubuntu(20.04, 22.04)

Pre-requisites

  1. Install Go as described at https://golang.org/doc/install

    • Ensure that the following line has been appended to /etc/profile:

      export PATH=$PATH:/usr/local/go/bin

    • Ensure that the following line has been appended to ~/.profile:

      export PATH=$PATH:~/go/bin

  2. Install Docker as described at https://docs.docker.com/engine/install/ubuntu/

    • In Installation methods section follow Install using the repository method

    • Check whether your user of Ubuntu has been added to docker group using the following command:

      getent group docker | awk -F: '{print $4}'

  3. Install Docker Compose as described at https://docs.docker.com/compose/install/

Build and test

  1. Building

    make build
make install

  2. Run unit tests

    make test

  3. Run integration tests.

    The integration tests are run against a local pool of nodes in Docker. REST integration tests need to have a backend running (CLI in REST mode).

    The following script will start all necessary things and run all the tests:

    ./integration_tests/run-all.sh

    If you want to run a particular group of tests (cli, light, rest, deploy, upgrade), you can

    ./integration_tests/run-all.sh cli
./integration_tests/run-all.sh light
./integration_tests/run-all.sh rest
./integration_tests/run-all.sh upgrade
./integration_tests/run-all.sh cli,light

    If you want to run a particular test you may:

    ./integration_tests/start-pool.sh

bash <path-to-shell-script>  # to run a cli test
# OR
go test <path-to-go-test-file> # to run REST or gRPC go test

  4. Run deployment test

    The deployment test verifies deployment steps described in docs/running-node.md.

    ./integration_tests/deploy/test_deploy.sh

Run local pool

The easiest way to run a local pool is to start it in Docker.

Validator nodes only (no Observers):

make install
make localnet_init
make localnet_start

Validator and Observer nodes:

make install
DCL_OBSERVERS=1 make localnet_init
make localnet_start

This will start a local pool of 4 validator nodes in Docker. The nodes will expose their RPC endpoints on ports 26657, 26659, 26661, 26663 correspondingly.

Stopping the network:

make localnet_stop

Then you can start the network again with the existing data using make localnet_start

If you need to start a new clean network then run make localnet_rebuild prior to executing make localnet_start. It will remove .dcl directories from your user home directory (~), remove .localnet directory from the root directory of the cloned project, and initialize a new network data using make localnet_init.

CLI

Start a local pool as described above, and then just execute

dcld

Have a look at How To and transactions for instructions how to configure and use the CLI.

REST

Start a local pool as described above.

Every node exposes a REST API at http://<node-host>:1317 (see https://docs.cosmos.network/sdk/v0.47/learn/advanced/grpc_rest).

Have a look at transactions for a full list of REST endpoints.

Contributing

Please take into account the following when sending a PR:

  1. Make sure the new functionality has unit tests added

  2. Make sure the new functionality has integration tests added

  3. There is CI based on GitHub Actions that will do the following for every Pull Request:

    • make sure the app can be built
    • run go linter
    • run unit tests
    • run integration tests

How To Add a new Module or Transaction

  • Use ignite v0.27.1 command to scaffold the module. Consider using a docker container built from the provided Dockerfile to have a predictable version of ignite. See README.md.

  • Have a look at the scripts and commands used for generation of existing modules, messages and CRUD operations and do it in a similar way (for example PKI module commands).

    Note:
    1. On previous scaffolding of modules starport cli used instead of ignite. While generating new module, command structure will the same except it must start with ignite.
    2. After execution of command ignite scaffold ... generated .proto files will be added into proto/distributedcoplianceledger/{module}/{proto} . So make sure to put contents of generated proto files into related folders inside proto/zigbeealliance/.. folder.
  • If a new transaction with a new data in state (key-value) and new queries needs to be created, then both message and CRUD commands need to be executed.
  • If just a message to update existing state values need to be created, then a message command is enough.
  • Adjust the generated code

    • increment the return value of AppModule.ConsensusVersion method in x/<module>/module.go

    • correct REST endpoints: /dcl instead of /zigbee-alliance/distributedcomplianceledger in proto/<module>/query.proto and in entries related to queries of the new module in docs/static/openapi.yml

    • add message validation as annotations (validate tags) in proto/<module>/tx.proto

    • add (cosmos_proto.scalar) = "cosmos.AddressString" annotation for all fields with address/account type (such as signer or owner).

    • fix types if needed in proto/<module>/<entity>.proto files

      • Note1: unit64 will be returned as string if the output is a JSON format. So, it's better to use uint64 only when it's really uint64.
      • Note2: for uint16 type: use int32 during ignite scaffolding, and add custom validation (annotations above) to check the lower and upper bounds.
      • Note3: for uint32 type: use int32 during ignite scaffolding, then replace it by uint32 in .proto files, re-generate the code and fix compilation errors.

    • build proto (for example ignite chain build). Fix compilation errors if any.

    • generate openapi docs from proto using (scripts/dcl-swagger-gen.sh). It's recommended to run from container built from Dockerfile

    • regenerate the TypeScript client via make ts-client-gen (see Regenerate the TypeScript Client).

    • Note1: colons (:) are part of subject-id in PKI module, but colons are not allowed in gRPC REST URLs by default. allow_colon_final_segments=true should be used as a workaround. So, make sure that runtime.AssumeColonVerbOpt(false) in /x/pki/types/query.pb.gw.go. It's usually sufficient to revert the generated changes in /x/pki/types/query.pb.gw.go. It may be easier just to revert changes in all *.pb.go files not affected by your changes in .proto

    • Note3: ignite chain build needs to be called only if you made manual changes in .proto files. There is no need to call ignite chain build again once all errors and adjustments above are done. It's sufficient just to build the project via usual ways (such as make build)

  • Add static validation for new messages:
    • Call validator.Validate(msg) in ValidateBasic methods for all generated messages
    • Add additional checks to ValidateBasic that do not depend on the state (key-value) and order of transactions
  • Implement business logic in msg_server_xxx.go
  • Improve NotFound error processing:
    • replace status.Error(codes.InvalidArgument, "not found") to status.Error(codes.NotFound, "not found") in every generated grpc_query_xxx.go to return 404 error in REST.
  • Support state proof for single value queries in CLI:
    • use cli.QueryWithProof instead of cosmos queries that doesn't support state proofs
    • add proper handling for list queries and write requests when used with a light client proxy (see IsWriteInsteadReadRpcError and IsKeyNotFoundRpcError)
  • Add unit tests (see other modules for reference)
  • Add CLI-based integration tests to integration_tests/cli/<module> (see other modules for reference)
  • Add gRPC/REST-based integration tests to integration_tests/grpc_rest/<module> (see other modules for reference)

How To Make Changes in Data Model for Existing Modules

  • Use ignite command to scaffold the module. Consider using the provided Dockerfile to have a predictable version of ignite. See README.md.

  • Never change .pb files manually. Do the changes in .proto files.

  • Every time .proto files change, re-generate the code (for example ignite chain build) and fix compilation errors if any.

  • Update openapi docs from proto using (scripts/dcl-swagger-gen.sh). It's recommended to run from container built from Dockerfile.

  • Regenerate the TypeScript client via make ts-client-gen (see Regenerate the TypeScript Client).

  • Note1: colons (:) are part of subject-id in PKI module, but colons are not allowed in gRPC REST URLs by default. allow_colon_final_segments=true should be used as a workaround. So, make sure that runtime.AssumeColonVerbOpt(false) in /x/pki/types/query.pb.gw.go. It's usually sufficient to revert the generated changes in /x/pki/types/query.pb.gw.go.

  • Note2: move compliance_info.pb.go and compliance_history_item.pb.go to types/compliance and adjust the import in other places accordingly. It may be easier just to revert changes in all *.pb.go files not affected by your changes in .proto

  • Note3: ignite chain build needs to be called only if you made manual changes in .proto files. There is no need to call ignite chain build again once all errors and adjustments above are done. It's sufficient just to build the project via usual ways (such as make build)

Regenerate the TypeScript Client

The TypeScript client under ts-client/ is generated from the .proto files via ignite generate ts-client. After regeneration, the RFC 3986 query-encoding patch must be re-applied so base64 pagination keys (which contain +, /, =) survive transit — otherwise the cosmos-sdk gRPC-gateway decodes + as space and rejects next_key follow-up requests with illegal base64 data.

Run the below command to regenerate the ts-client and patch it:

make ts-client-gen

The patch script re-creates ts-client/utils.ts (which exports the shared paramsSerializer) and rewrites each ts-client/*/rest.ts to wire that serializer into the axios instance. Always run make ts-client-gen (never ignite generate ts-client on its own) when regenerating, and commit both the generated files and the patched ones together.

To verify the patch is in place:

make ts-client-test

Update Cosmos-sdk Version

Re-generate cosmos base openapi (service API from cosmos exposed in DCL) using cosmos-swagger-gen.sh from the project root:

  • Consider using a docker container built from the provided Dockerfile to have a predictable version of swagger
./scripts/cosmos-swagger-gen.sh base
./scripts/cosmos-swagger-gen.sh tx

Update CometBFT Version

Please note, that we depend on the CometBFT fork https://github.com/zigbee-alliance/cometbft/releases/tag/v0.37.5 due to hotfixes for tendermint/tendermint#7640 and tendermint/tendermint#7641 required for Light Client Proxy.

Also don't forget to update the link to the CometBFT RPC in Swagger UI.

Other

For more details, please have a look at Cosmos SDK tutorial.