Attribute prediction task

We support the goal-oriented evaluation with the attribute prediction auxiliary task related to assessing the degree of compositionality of the representations learned for a specific task. With an attribute prediction task, we can assess whether the learned representations capture what we think they should, in terms of object attributes, rather than spurious correlations.

In the context of guessing game, we regard the representation for the last turn of the dialogue as a composition or aggregation of all the attributes specified so far. Therefore, we can use it to predict with high accuracy the attributes associated with a specific target object because it should encode the information needed to correctly discriminate the target from all the other objects in the scene. For instance, by playing several guessing games that have a microwave as the target object, the agent should learn a representation of microwave that is expressive enough to correctly discriminate a microwave from all the other objects in a scene.

For the attribute prediction task, we extract different multi-modal and amodal representations coming from state-of-the-art models for GuessWhat?! and we train a linear classifier for each representation so as to learn to predict the attributes of a given target object. In order to reproduce our experiments, you can use our probing classifiers framework in the GitHub repository comb_probing.

Previous Post Next Post