Some NLP: Probabilistic Context Free Grammar (PCFG) and CKY Parsing in Python

For this assignment we will use your average F1 score to evaluate the correctness of the CKY parser, although in essence we ought to know from the output on the development set (devtest) whether the parser is implemented correctly. The following figure shows all the training trees. There are just enough productions in the training set for the test sentence to have an ambiguity due to PP-attachment. The following figure shows the PCFG learnt from these training trees. Now let's try to parse a single test sentence'cats scratch walls with claws' with the CKY parser and using the PCFG grammar learnt.