Okay fags so heres the thing.
In Latent dirichlet allocation, one of the hyperparameters, dirichlet prior on word distribution, given any topic is of dimension equal to the size of the vocabulary of the corpus.
Suppose i was to optimize the hyperparameters and number of topics using grid search. I am not sure what i should do if i want to implement cross validation and testing of the model on held out set of data, in regards to said prior hyperparameter- one could do grid search to optimize by some metric the model in terms of said prior, but if i do not include a naive assumption, that all the train set(for cross validation) and test set (final evaluation of the model) share the same vocabulary, then it seems that i cannot perform some detailed optimizaton on the word topic distribution prior, since the vocabulary inferred each time from training set changes the dimension of the hyperparameter i seek to optimize (unless i assume symmetric prior and repeat the same value V times, where V is the size of each vocab).
I could probably get away with such assumption for my paper and skip the details and no one would care, but I actually wonder wtf is going on with the vocabary in rigorous treatment of the evaluation of the model and i cannot seem to figure this out, and papers never address that thing.
In Latent dirichlet allocation, one of the hyperparameters, dirichlet prior on word distribution, given any topic is of dimension equal to the size of the vocabulary of the corpus.
Suppose i was to optimize the hyperparameters and number of topics using grid search. I am not sure what i should do if i want to implement cross validation and testing of the model on held out set of data, in regards to said prior hyperparameter- one could do grid search to optimize by some metric the model in terms of said prior, but if i do not include a naive assumption, that all the train set(for cross validation) and test set (final evaluation of the model) share the same vocabulary, then it seems that i cannot perform some detailed optimizaton on the word topic distribution prior, since the vocabulary inferred each time from training set changes the dimension of the hyperparameter i seek to optimize (unless i assume symmetric prior and repeat the same value V times, where V is the size of each vocab).
I could probably get away with such assumption for my paper and skip the details and no one would care, but I actually wonder wtf is going on with the vocabary in rigorous treatment of the evaluation of the model and i cannot seem to figure this out, and papers never address that thing.
