README.md (14796B)
1 # chainlib 2 3 # Overview 4 5 Chainlib is an attempt at employing a universal interface to manipulate 6 and access blockchains regardless of underlying architecture. 7 8 It makes the following assumptions: 9 10 - A block MUST have a interpretable serializable format, and contains 11 zero of more transactions 12 13 - A transaction MUST have a interpretable serializable format 14 15 - A transaction MUST have a nonce associated with a sender address. This 16 uniquely identifies the transaction on the network. 17 18 - A transaction MUST have a fee bid to get the transaction executed on 19 the network (a network that does not care about bids can just ignore 20 this property). 21 22 - A transaction signature MAY be locked to a particular chain identifier 23 24 - The sender key of a transaction MAY be recovered by the signature of 25 the transaction 26 27 Chainlib is specifically designed for building command line interface 28 tools. It provides templates for handling configuration, argument 29 parsing and environment variable processing to build RPC connections to 30 chain nodes for network queries and signing operations. 31 32 # Command line interface provisions 33 34 The base CLI provisions of `chainlib` simplifies the generation of a 35 some base object instances by command line arguments, environment 36 variables and configuration schemas. 37 38 To use CLI provisions, `chainlib.cli` should be imported. This 39 automatically imports the following submodules: 40 41 `arg` 42 Define and/or select command-line arguments 43 44 `config` 45 Process configuration from command-line arguments and environment 46 variables 47 48 `rpc` 49 Create RPC connection from configuration 50 51 `wallet` 52 Create wallet from configuration 53 54 Any chain implementation building on `chainlib` should extend one or 55 more of the classes in these modules as needed, for example order to add 56 more configuration directives or command line argument flags. 57 58 ## Arguments 59 60 `chainlib` defines a set of arguments that are common concepts for 61 interfacing with blockchain RPCs. Which arguments to use for a specific 62 instantiation can be defined using flags or symbols that define groups 63 of flags. 64 65 This functionality is provided by the `chainlib.cli.args.ArgumentParser` 66 class. It is a thin wrapper around the standard library 67 `argparser.ArgumentParser` class, only adding a method to add arguments 68 to the instance based on the aforementioned flags. 69 70 Following is a description of all pre-defined arguments that are 71 available with `chainlib`. 72 73 ### -c, –config 74 75 Override configuration directives by reading ini-files in the given 76 directory. 77 78 Only configuration directives defined in the schema may be overridden. 79 See [chainlib-config](#chainlib_002dconfig). 80 81 ### –env-prefix 82 83 Prepend the given string to configuration directives when overriding by 84 environment variables 85 86 Normally, if a configuration directive `FOO_BAR` exists, the environment 87 variable `FOO_BAR` will override its value. If `--env-prefix BAZ` is 88 passed, the environment variable `BAZ_FOO_BAR` will be used instead to 89 override the configuration directive `FOO_BAR`. The environment variable 90 `FOO_BAR` will in this case *not* be used. 91 92 ### –height 93 94 Query the chain RPC for results at a specific block height. 95 96 Applies to *read* operations only. 97 98 ### -i, –chain-spec 99 100 Chain specification string for the blockchain connection. 101 102 This informs the implementing code about the architecture and deployment 103 of the blockchain network. It can also be relevant when creating 104 signatures for the network (e.g. the EIP155 signature scheme for EVM). 105 106 ### –fee-limit 107 108 Use the exact given fee multiplier to calculate the final bid to get 109 transaction executed on the network. 110 111 How the fee semantics are employed depend on the chain implementation, 112 but the final resulting bid *must always* be the product of 113 `price * limit`. 114 115 If *not* defined, the multiplier will be retrieved using the fees 116 provider defined by the implementation. 117 118 ### –fee-price 119 120 Use the exact given fee price as factor to calculate bid to get 121 transaction executed on the network. 122 123 How the fee semantics are employed depend on the chain implementation, 124 but the final resulting bid *must always* be the product of 125 `price * limit`. 126 127 If *not* defined, the current recommended price will be retrieved from 128 the fees provider defined by the implementation. 129 130 ### -n, –namespace 131 132 Append the given namespace to implicit configuration override paths. 133 134 For example, if linux xdg-basedir path is used, a namespace argument of 135 `foo` in implementation domain `bar` will result in the configuration 136 override path `$HOME/.config/bar/foo`. 137 138 ### –nonce 139 140 Start at the exact given nonce for the query. 141 142 If *not* defined, the next nonce will be retrieved from the nonce 143 provider defined by the implementation. 144 145 ### -p, –provider 146 147 URL of the chain RPC provider. 148 149 ### -s, –send 150 151 CLI tools building on chainlib should *never* submit to the network by 152 default. Instead, resulting transactions ready for network submission 153 should be output to terminal. 154 155 If the implementation wishes to allow the user to directly send to the 156 network, the `-s` flag *must* be used for this purpose. 157 158 ### –seq 159 160 By default, a random RPC id will be generated for every RPC call. 161 162 However, some RPCs will only allow sequential serial numbers to be used 163 as RPC ids, in which case this flag should be used. 164 165 ### –raw 166 167 Generate output suitable for passing to another command (e.g. UNIX 168 pipes). 169 170 ### –rpc-auth 171 172 Tells the implementer which RPC authentication scheme to use (e.g. 173 "basic" for http basic). 174 175 ### –rpc-credentials 176 177 Tells the implemented wich RPC authentication credentials to use for 178 selected rpc authentication scheme (e.g. "foo:bar" for user foo pass bar 179 in scheme "basic" a.k.a. http basic). 180 181 Credentials may for example also be a single value, like a private key, 182 depending on the scheme and implementation. 183 184 ### –rpc-dialect 185 186 Tells the implementer to optimize query, result and error reporting for 187 the specific chain RPC backend dialect. 188 189 ### -u, –unsafe 190 191 Allow arguments with blockchain addresses that are not checksum 192 protected. 193 194 ### -v, -vv 195 196 Defines logging verbosity. 197 198 Specifically, `-v` will set loglevel to `INFO`, wheres `-vv` will set 199 loglevel to `DEBUG`. 200 201 Default loglevel is up to the implementer, but it is advisable to keep 202 it at `WARNING`. 203 204 ### -w, -ww 205 206 Toggles blocking in relation to chain RPC calls. 207 208 If `-w` is set, the implementer should only block to obtain the result 209 of the *last, and as few as possible preceding* RPC transactions. 210 211 If `-ww` is set, the implementer should block to retrieve the results of 212 *all* of the preceding RPC transactions. 213 214 If the implementation consists of a single transaction, the effect of 215 `-w` and `-ww` will always be the same. Nonetheless, the implementation 216 will be forced to provide both arguments. 217 218 If neither flag is set, the typical consequence is that the network 219 transaction hash of the last transaction will be returned. 220 221 ### -y, –key-file 222 223 Read private key from the given key file. 224 225 # Rendering configurations 226 227 Configurations in `chainlib` are processed, rendered and interfaced 228 using the `confini` python package. 229 230 In short, `confini` extends the functionality of Python’s standard 231 library `configparser` package to allow merging of directives by parsing 232 multiple ini-files from multiple directories. 233 234 Furthermore, it employs this same multi-file approach when defining a 235 configuration *schema*, aswell as defining source to *override* the 236 values defined in the schema. 237 238 See <https://gitlab.com/nolash/python-confini> for further details on 239 `confini`. 240 241 ## Configuration schema in chainlib 242 243 The chainlib configuration schema is, along with the provided command 244 line arguments and environment variables, designed to cover the settings 245 necessary for most normal chain RPC operations. 246 247 ### Configuration mapping 248 249 Below follows a mapping of configuration directives to command line 250 arguments and environment variables. 251 252 Note that any configuration directives prefixed by `"_"` are not defined 253 in the configuration schema, and are not overrideable by environment 254 variables. These are values typically are only valid within the context 255 of a single execution of the implemented tool. 256 257 | arg | config | env | 258 |-------------------------|-----------------------|-----------------------| 259 | — | — | CONFINI_DIR [^1] | 260 | -c. –config [^2] | — | — | 261 | -i, –chain-spec | CHAIN_SPEC | CHAIN_SPEC | 262 | –height | \_HEIGHT | — | 263 | –fee-limit | \_FEE_LIMIT | — | 264 | –fee-price | \_FEE_PRICE | — | 265 | -n, –namespace | CONFIG_USER_NAMESPACE | CONFIG_USER_NAMESPACE | 266 | –nonce | \_NONCE | — | 267 | -p, –rpc-provider | RPC_HTTP_PROVIDER | RPC_HTTP_PROVIDER | 268 | -a, -recipient | \_RECIPIENT | — | 269 | -e, –executable-address | \_EXEC_ADDRESS | — | 270 | –rpc-auth | RPC_AUTH | RPC_AUTH | 271 | — | RPC_PROVIDER | RPC_PROVIDER [^3] | 272 | — | RPC_SCHEME | RPC_SCHEME [^4] | 273 | –rpc-credentials | RPC_CREDENTIALS | RPC_CREDENTIALS | 274 | –rpc-dialect | RPC_DIALECT | RPC_DIALECT | 275 | -s, –send | \_RPC_SEND | — | 276 | –seq | \_SEQ | — | 277 | -u, –unsafe | \_UNSAFE | — | 278 | -w | \_WAIT | — | 279 | -ww | \_WAIT_ALL | — | 280 | -y, –key-file | WALLET_KEY_FILE | WALLET_KEY_FILE | 281 | — [^5] | WALLET_PASSPHRASE | WALLET_PASSPHRASE | 282 283 Currently `chainlib` only supports HTTP(S) when building RPC connections 284 from command line arguments and configuration. 285 286 # Base library contents 287 288 ## Pluggable method interface 289 290 The base chainlib blockchain interface is defined by the 291 `chainlib.interface.ChainInterface class`. All of the methods in this 292 class are unimplemented. Together they make up the methods necessary to 293 interface with *any* blockchain RPC. 294 295 It is up to the implemenenter to choose which of the methods that are 296 needed in any particular context. The implementer would then connect the 297 method symbols with actual code. 298 299 Most methods in this class will return objects that can be passed to an 300 RPC connection that fits the block context. 301 302 The available methods are: 303 304 `block_latest` 305 Retrieve the latest block from the network 306 307 `block_by_hash` 308 Retrieve the block corresponding to the given block hash 309 310 `block_by_number` 311 Retrieve the block corresponding to the given block number 312 313 `block_from_src` 314 Render a chainlib.block.Block derivative object from an 315 architecture-dependent block representation source 316 317 `block_to_src` 318 Render an architecture dependent transaction representation from the 319 given Block object 320 321 `tx_by_hash` 322 Retrieve the transaction corresponding to the given transaction hash 323 324 `tx_by_block` 325 Retrieve the transaction corresponding to the given block hash and 326 transaction index 327 328 `tx_receipt` 329 Retrieve the details of a confirmed transaction 330 331 `tx_raw` 332 Generate an RPC query from raw transaction wire data 333 334 `tx_pack` 335 Generate raw transaction wire data from an architecture dependent 336 transaction representation 337 338 `tx_unpack` 339 Generate architecture dependent transaction representation from raw 340 transaction wire data 341 342 `tx_from_src` 343 Render a chainlib.tx.Tx derivative object from an architecture-dependent 344 tx representation source 345 346 `tx_to_src` 347 Render an architecture dependent transaction representation from the 348 given Tx object 349 350 `address_safe` 351 Generate a checksum-safe network address 352 353 `address_normal` 354 Generate an unambiguous network address 355 356 `src_normalize` 357 Generate an unambiguous dictionary from the given dictionary. For 358 example, this can mean generating camel-case key equivalents for 359 snake-case values. 360 361 ## The RPC interface 362 363 `chainlib.connection` currently has support for HTTP(S) and UNIX socket 364 RPC connections. Both rely on the Python *standard library* only 365 (`urllib` and `socket`). 366 367 It provides a thread-safe connection factory mechanism where connection 368 constructor and location pairs are associated with string labels. 369 370 There is also explicit builtin support for the JSONRPC RPC protocol, 371 which allows for a pluggable error translater that can be customized to 372 every RPC "dialect" that needs to be supported (examples are "geth" and 373 "openethereum" dialects of the Ethereum node fauna). Classes to handle 374 JSONRPC results, requests and errors are defined in the 375 `chainlib.jsonrpc` module. 376 377 ## Blocks and transactions 378 379 Common block and transaction concepts are represented by the 380 `chainlib.block.Block` and `chainlib.tx.Tx` objects. These are very 381 minimal base-classes that need to be extended for every blockchain 382 implementation that is to be supported. 383 384 When building transactions, implementations of the 385 `chainlib.sign.Signer`, `chainlib.nonce.NonceOracle` and 386 `chainlib.fee.FeeOracle` interfaces will provide the transaction factory 387 object of the implementation with signatures, transaction nonces and 388 transaction fee details respectively. 389 390 ## Other code features 391 392 This section lists features that are considered outside the core of the 393 `chainlib` package 394 395 ### RPC authenticator 396 397 If you are relying on an RPC provider instead of running your own node 398 (although, you know you *should* run your own node, right?), then RPC 399 authentication may be relevant. 400 401 `chainlib.auth` provides two authentication mechanisms for HTTP: 402 403 `BasicAuth` 404 The HTTP basic Authorization scheme 405 406 `CustomHeaderTokenAuth` 407 Define an arbitrary header name and value 408 409 ### Fee price aggregator 410 411 The `chainlib.stat.ChainStat` class provides a simple implementation of 412 a running average aggregator for network fee prices. This can be used to 413 generate more precise fee price heuristics that in turn can be fed to a 414 Fee Oracle. 415 416 [^1]: The `CONFINI_DIR` environment variable defines an explicit 417 configuration *schema* path. 418 419 [^2]: The `-c` flag defines an explicit configuration *override* path. 420 421 [^3]: RPC_PROVIDER will always be set to the same value as 422 RPC_HTTP_PROVIDER by default. 423 424 [^4]: The RPC_SCHEME configuration directive is always set to ’http’ by 425 default. 426 427 [^5]: This flag is not provided because it is simply not safe to define 428 passphrases as an argument on the command line.