Btcd + getwork + cgminer = profit

We are pleased to announce that btcd, our full-node bitcoind (bitcoin core) alternative written in Go, now has support for the getwork RPC which allows it to function with cgminer!

We have extensively tested the code on testnet. The following is a sampling of the latest blocks we’ve generated on testnet using btcd + cgminer: 226713 226830 227228 227390 227393.

If you are simply interested in learning how to configure btcd to work with cgminer, you can view the setup instructions on the btcd wiki, however if you’re interested to learn about some of the nuts and bolts that make it all work, read on.

Isn’t getwork deprecated?

Before I dig into some of the details, I would like to touch on a topic that I’m sure some astute readers will undoubtedly be asking, which is “Why implement getwork when it has been deprecated in favor of getblocktemplate (BIP0022 and BIP0023)?”

There are a few reasons why we chose to implement it first:

• Supporting getwork was a lot less time consuming than getblocktemplate proposals and software such as cgminer still functions properly with getwork. This means we were able to release support for mining more quickly by supporting getwork first.
• The bulk of the work involved was the code to create and manipulate block templates which is used by both getwork and getblocktemplate. Thus it helps pave the way to supporting getblocktemplate proposals while allowing getwork to function in the mean time.
• The current reference implementation (bitcoin core) still supports getwork. Even though it will likely be removed in the next version, one of the goals of btcd is to be a drop-in alternative that can be used with existing infrastructures.

Overview of the getwork Process

The getwork RPC has two modes. A request mode and a submission mode. The request mode instructs the RPC server to return data for the caller to work on. The submission mode accepts a supposedly solved block header from the caller and checks it for accuracy, adds it to end of block chain following the same rules as every other block, and relays it to rest of the network.

A high-level overview of the getwork process is as follows:

• The caller (mining software such as cgminer) submits a request for work by not specifying the optional data parameter on the getwork RPC
• The server generates a block template which extends the current best chain using the available transactions in the memory pool and several other factors which are discussed further in the Block Templates section
• The server returns the block header in a padded form with the intention of allowing optimized hashing and other information to this end as detailed in the Getwork Request section
• The caller attempts to solve the block by finding a nonce which allows the double sha256 of the block to fall under the required target
• If the caller is unable to find a nonce which solves the block, it issues another request for more work and repeats the process until it finds a solution
• Once the caller finds a solution, it updates the nonce in the block header and submits it by calling getwork with the completed block header (again using the padded form) as detailed in the Getwork Submission section
• The server validates the submission by following the same rules every other block is subjected to, adds it to the end of the block chain, and relays it to the rest of the network if validation was successful.

What address gets paid?

Something you might have noticed in the overview above is the address to pay the generated bitcoins to is not specified. In the reference implementation, this is not an issue because the wallet and chain functionality is tightly integrated. However, btcd intentionally separates the wallet and chain functionality into separate processes.

As a consequence, btcd needs to be instructed which address to pay to in the generated block template’s coinbase. This is accomplished via the getworkkey option which can be specified multiple times. When there are multiple addresses, they will be rotated through at random when creating block templates.

Block Templates

One of the fundamental pieces needed to support getwork, and getblocktemplate in the future, is block templates. The following diagram shows the layout of a block template that btcd generates:

Block template creation is an intricate process that involves prioritizing transactions based on factors such as their fees per kilobyte, value, age of inputs, and byte size. Additionally, the code has to ensure that the selected transactions don’t run afoul of consensus rules. Some examples are exceeding maximum signature operations per block, exceeding maximum block size, not being finalized, and having dependencies which are not already in the block chain or earlier in the block being constructed.

By default, the block template created by btcd contains a configurable area which includes transactions that are considered high priority. Transactions which consist of large monetary amounts, old inputs, and small byte sizes have the highest priority. Among other things, this helps ensure low-fee transactions that have been around for a while eventually rise in priority to the point they are more likely to be included in a block regardless of outstanding transactions with higher fees.

Once the high-priority area is filled, or the priority falls sufficiently low, the transactions are re-prioritized from the highest fees per kilobyte to the lowest in preparation to fill the next area.

The next thing that happens depends on whether or not there is a minimum block size configured (there is not one by default). If unconfigured, the block template generation is complete as soon as the transaction fees fall below a minimum threshold that is deemed too low. If configured, transactions will continue to be selected in descending fee order until the minimum block size is reached.

At any time during this process, the block template generation is complete if there are no more transactions or the maximum block size is reached.

getwork Request

As previously mentioned, when a getwork call is made without the optional data parameter, it acts as a request for work.

The following is an overview of processing a getwork request:

• Determine if a new full block template needs to be generated based on these three factors:
  • The end of the main chain has changed since the last call
  • The transactions in the memory pool have been updated since the last call
  • It has been at least one minute since the last template was generated
• Otherwise, if a new block template does not need to be generated:
  • Update the block header’s timestamp to the current time while accounting for median time consensus requirement
  • Increment the extra nonce in the coinbase script
• Cache the block template and its variations by their merkle root
• Calculate and format the data as this example shows:

{
  "data":     "...e8f7356a1efce26b24bd15d7d906e85341ef9cec99b6a000000006474f...",
  "hash1":    "...0000000000000000000000000000000000000000000000000000000800...",
  "midstate": "ae4a80fc51476e452de855b4e20d5f33418c50fc7cae3b1ecd5badb819b8a584",
  "target":   "0000000000000000000000000000000000000000000000008c96010000000000",
}

As you might have noticed from the overview, one of the interesting properties about the getwork RPC is that unlike most others, it maintains state between calls. This is necessary because the actual transactions which are selected to form the block template are never provided to the caller since they are incorporated into the merkle root hash. This means they need to be cached for retrieval later when the caller eventually submits a solution.

Since there will usually be a lot of outstanding block template variations before a solution is found, each variation is efficiently cached by its merkle root with a pointer to the original block template and the mutated coinbase script which contains the incremented extra nonce. This provides enough information for the block to be reconstructed based on the data submitted by the caller in the Getwork Submission.

getwork Submission

Finally, when getwork is called with the optional data parameter specified, it is treated as a submission of a valid solution.

The following is an overview of processing a getwork submission:

• Look up the cached block template variation based on the merkle root in the submitted block header
• Reconstruct the block based on the cached data and the submitted block header
• Ensure the submitted block header hash is less than the target difficulty as a quick check to weed out erroneous submissions
• Return false if the block is stale (another valid block was relayed while the caller was working to find a valid solution)
• Return false if the block fails to validate or connect to the main chain using the same rules every other block is subjected to
• Relay the block the rest of the network

What’s Next?

Adding support for getblocktemplate is a high priority for us (Issue #124). Our first goal is to implement and release the getblocktemplate fundamentals (BIP0022) and then later add support for the more advanced getblocktemplate features detailed by BIP0023.

Also, we are looking for miners with ASIC mining hardware that are willing to incorporate btcd into their infrastructure. So, if you are interested, feel free to contact us to discuss options.