Lemma

Lemma · No. 0002

accepted

Accepted on Bradbury. The appeal and finalization window is still open.

Verdict

verified

The claim is faithfully supported by the cited source. The validators reached consensus on this label.


The claim

BERT is pre-trained using a masked language model objective and a next sentence prediction objective.

Source

arxiv.org/abs/1810.04805

Abstract


Jury opinion

The source excerpt explicitly states BERT is "designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers," which is the core of the masked language modeling objective, and the original BERT paper is well-known for introducing the next sentence prediction task.


Submitted by

0x7713…0aDa

Stake consumed

0.001 GEN

Issue number

No. 0002

Appeals

0 / 1

Chain state

accepted


Permanent reference

lemma://verdict/0x59643615023fe63399f33a66d08af01669a59aa81eec2200b07307b86e91d750

Set in Newsreader & IBM Plex Mono · Printed on cream · MMXXVI