README.md (8659B)
1 # chaind-eth development tester recipe 2 3 chaind-eth is a socket server that acts as a automated transaction handler for an EVM network. 4 5 It capabilities are (unchecked box means feature not yet completed): 6 7 - [x] unix socket server to accept raw, signed RLP evm transactions 8 - [x] stateful queueing system following full local and remote lifecycle of the transaction 9 - [x] transaction dispatcher unit 10 - [x] transaction retry unit (for errored or suspended transactions) 11 - [x] blockchain listener that updates state of transactions in queue 12 - [x] CLI transaction listing tool, filterable by: 13 * [x] transaction range with lower and/or upper bound 14 * [x] only show transaction with errors 15 * [x] only show transaction that have not yet completed 16 - [x] systemd unit / socket service 17 - [x] sql storage backend 18 - [x] filesystem storage backend 19 20 21 ## prerequisites 22 23 For these examples you need: 24 25 - linux (tested on 5.12.x, perhaps wsl/macos will work too, no guarantees, though) 26 - python 3.9.x 27 - pip 28 - virtualenv 29 - socat 30 - sqlite 31 - an EVM RPC endpoint 32 33 For any python command / executable used below: 34 35 * add `-v` or `-vv` to get more information about what is going on 36 * use with `--help` for information on how to use and parameters that can be passed 37 38 39 ## setting up the database backend 40 41 Currently there is no more practical way of setting up the database backend than to pull the repository and run a database migration script :/ 42 43 ``` 44 git clone https://git.defalsify.org/chaind 45 cd chaind 46 python -m venv .venv 47 . .venv/bin/activate 48 pip install --extra-index-url https://pip.grassrootseconomics.net -r requirements.txt 49 # the following will set up your database in ~/.local/share/chaind/eth/chaind.sqlite 50 PYTHONPATH=. CHAIND_DOMAIN=eth DATABASE_ENGINE=sqlite python scripts/migrate.py 51 ``` 52 53 54 ## usage example 55 56 ### create an empty working directory 57 58 In terminal window A 59 60 ``` 61 d=$(mktemp -d) && cd $d 62 ``` 63 64 ### create a chaind-eth sandbox 65 66 ``` 67 python -m venv .venv 68 . .venv/bin/activate 69 pip install --extra-index-url https://pip.grassrootseconomics.net "chaind-eth>=0.0.3a5" 70 ``` 71 72 ### start the services 73 74 In terminal window B 75 76 ``` 77 cd <working directory> 78 . .venv/bin/activate 79 export DATABASE_ENGINE=sqlite 80 export RPC_PROVIDER=<your_provider> 81 export CHAIN_SPEC=<chain_spec_of_provider> 82 chaind-eth-server --session-id testsession 83 ``` 84 85 In terminal window C 86 87 ``` 88 cd <working directory> 89 . .venv/bin/activate 90 export DATABASE_ENGINE=sqlite 91 export RPC_PROVIDER=<your_provider> 92 export CHAIN_SPEC=<chain_spec_of_provider> 93 chaind-eth-syncer 94 ``` 95 96 ### prepare test transactions 97 98 Create two transactions from sender in keyfile (which needs to have gas balance) to a newly created account 99 100 ``` 101 export WALLET_KEY_FILE=<path_to_keyfile> 102 export WALLET_PASSWORD=<keyfile_password_if_needed> 103 export RPC_PROVIDER=<your_provider> 104 export CHAIN_SPEC=<chain_spec_of_provider> 105 106 # create new account and store address in variable 107 eth-keyfile -z > testkey.json 108 recipient=$(eth-keyfile -z -d testkey.json) 109 110 # create transactions 111 eth-gas --raw -a $recipient 1024 > tx1.txt 112 eth-gas --raw -a $recipient 2048 > tx2.txt 113 eth-gas --raw -a $recipient 4096 > tx3.txt 114 ``` 115 116 ### send test transactions to queue 117 118 ``` 119 cat tx1.txt | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock - 120 cat tx2.txt | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock - 121 cat tx3.txt | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock - 122 ``` 123 124 ### check status of transactions 125 126 127 `chainqueue-list` outputs details about transactions in the queue. 128 129 Provided the initial database migration was executed as described above, the execution would look as follows: 130 131 ``` 132 export DATABASE_ENGINE=sqlite 133 export DATABASE_NAME=$HOME/.local/share/chaind/eth/chaind.sqlite 134 export CHAIN_SPEC=<chain_spec_of_provider> 135 sender=$(eth-keyfile -d $WALLET_KEY_FILE) 136 chainqueue-list $sender 137 ``` 138 139 To show a summary only instead all transactions: 140 141 ``` 142 chainqueue-list --summary $sender 143 ``` 144 145 The `chaind-list` tool can be used to list by session id. Following the above examples: 146 147 ``` 148 export DATABASE_ENGINE=sqlite 149 export CHAIN_SPEC=<chain_spec_of_provider> 150 chaind-list testsession 151 ``` 152 153 The `chainqueue-list` and `chaind-list` tools both provides the same basic filtering. Use `--help` to see the details. 154 155 156 ### Retrieve transaction by hash 157 158 The socket server returns the transaction hash when a transaction is submitted. 159 160 If a socket server is given a transaction hash, it will return the transaction data for that hash (if it exists). 161 162 Extending the previous examples, this will output the original signed transaction: 163 164 ``` 165 eth-gas --raw -a $recipient 1024 > tx1.txt 166 cat tx1.txt | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock - | cut -b 4- > hash1.txt 167 cat hash1.tx | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock - | cut -b 4- > tx1_recovered.txt 168 diff tx1_recovered.txt tx1.txt 169 # should output 0 170 echo $? 171 ``` 172 173 The first 4 bytes of the data returned from the socket is a 32-bit big-endian result code. The data payload follows from the 5th byte. 174 175 176 ## Batch processing 177 178 The `chaind-eth-send` executable generates signed transactions with data from a csv file. 179 180 The data columns must be in the following order: 181 182 1. receipient address 183 2. transaction value 184 3. token specifier (optional, network fee token if not given) 185 4. network fee token value (optional) 186 187 188 If the gas token value (4) is not given for a gas token transaction, the transaction value (2) will be used. 189 190 By default the signed transactions are output as hex to stdout, each on a separate line. 191 192 If a valid `--socket` is given (i.e. the socket of the `chaind-eth-server`) the transactions will be send to the socket instead. The hash of the transaction will be output to standard output. 193 194 195 ### Using token symbols 196 197 If token symols are to be used in some or all values of column 3, then a valid `--token-index` executable address is required (in this case, a smart contract implementing the [`registry`](https://gitlab.com/cicnet/eth-contract-registry/-/blob/master/solidity/Registry.sol) contract interface). 198 199 200 ### Input validity checks 201 202 The validity of the input data is verified _before_ actual execution takes place. 203 204 These checks include: 205 206 - The token can be made sense of. 207 - The values can be parsed to integer amounts. 208 - The recipient address is a valid checksum address. 209 210 The checks do however _not_ include whether the token balances of the signer are sufficient to successfully execute the transactions on the network. 211 212 213 ### CSV input example 214 215 ``` 216 0x72B70906fD07c72f2d96aAA250C2D31662D0d809,10,0xb708175e3f6Cd850643aAF7B32212AFad50e2549 217 0xD536CB6d1d9B8d33875E0ba0Aa3515eD7478f889,0x2a,GFT,100 218 0xeE08b59a95E822AE346489038D25750C8EdfcC25,0x029a 219 ``` 220 221 This will result in the following transactions: 222 223 1. send 10 tokens from token contract `0xb708175e3f6Cd850643aAF7B32212AFad50e2549` to recipient `0x72B70906fD07c72f2d96aAA250C2D31662D0d809`. 224 2. send 42 `GFT` tokens along with 100 network gas tokens to recipient `0xD536CB6d1d9B8d33875E0ba0Aa3515eD7478f889` 225 3. send 666 network gas tokens to recipient `0xeE08b59a95E822AE346489038D25750C8EdfcC25` 226 227 228 ### Resending transactions 229 230 Since the `chaind-eth-server` does not have access to signing keys, resending stalled transactions is also a separate external action. 231 232 The `chaind-eth-resend` executable takes a list of signed transactions (e.g. as output from `chaind-eth-send` using the socket) and automatically increases the fee price of the transaction to create a replacement. 233 234 As with `chaind-eth-send`, the resend executable optionally takes a socket argument that sends the transaction directly to a socket. Otherwise, the signed transactions are send to standard output. 235 236 For example, the following will output details of the transaction generated by `chaind-eth-resend`, in which the fee price has been slightly incremented: 237 238 ``` 239 eth-gas --raw --fee-price 100000000 -a $recipient 1024 > tx1.txt 240 chaind-eth-resend tx1.txt > tx1_bump.txt 241 cat tx1_bump.txt | eth-decode 242 ``` 243 244 245 ### Retrieving transactions for resend 246 247 The `chaind-list` tool can be used to retrieve transactions with the same filters as `chainqueue-list`, but also allowing results limited a specific session id. 248 249 As with `chainqueue-list`, which column to output can be customized. This enables creation of signed transaction lists in the format accepted by `chaind-eth-resent`. 250 251 One examples of criteria for transactions due to be resent may be: 252 253 ``` 254 # get any pending transaction in session "testsession" 255 export DATABASE_ENGINE=sqlite 256 chaind-list -o signedtx --pending testsession 257 ``` 258 259 Note that the `chaind-list` tool requires a connection to the queueing backend. 260 261 262 ## systemd 263 264 TBC