Reposted by James Hutson
James Hutson
@jameshutson.bsky.social
3.1K followers 2.5K following 2.1K posts
Occasional illustrator, former animator of science communication explainers and all round information visualiser.
Occasional illustrator, former animator of science communication explainers and all round information visualiser.
![ENVIRONMENTAL AND FINANCIAL COST
Strubell et al. recently benchmarked model training and development costs in terms of dollars and estimated 퐶푂2 emissions [129].
While the average human is responsible for an estimated 5t 퐶푂2푒
per year,2
the authors trained a Transformer (big) model [136] with
neural architecture search and estimated that the training procedure
emitted 284t of 퐶푂2. Training a single BERT base model (without
hyperparameter tuning) on GPUs was estimated to require as much
energy as a trans-American flight.
While some of this energy comes from renewable sources, or
cloud compute companies’ use of carbon credit-offset sources, the
authors note that the majority of cloud compute providers’ energy is
not sourced from renewable sources and many energy sources in the
world are not carbon neutral. In addition, renewable energy sources
are still costly to the environment,3
and data centers with increasing
computation requirements take away from other potential uses of](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:66lbtw2porscqpmair6mir37/bafkreidy7oi2vznvk3xbvoscms4e2pds54pwv6k3aqhde2ezqiy7cy75i4)
![When we perform risk/benefit analyses of language technology,
we must keep in mind how the risks and benefits are distributed,
because they do not accrue to the same people. On the one hand, it
is well documented in the literature on environmental racism that
the negative effects of climate change are reaching and impacting
the world’s most marginalized communities first [1, 27].6
Is it fair or
just to ask, for example, that the residents of the Maldives (likely to
be underwater by 2100 [6]) or the 800,000 people in Sudan affected](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:66lbtw2porscqpmair6mir37/bafkreiab7kewjfrvolalvvyndkd4ax2ndpd7grjak5hr6khykjhfkgmpdq)
