The joint model for observable quantities and latent parameters is the starting point for Bayesian inference. It is challenging to specify such a model so that it both accurately describes the phenomena being studied, and is compatible with the available data. In this thesis we address challenges to model specification when we have either multiple data sources and/or expert knowledge about the observable quantities in our model.

We often collect many distinct data sets that capture different, but partially overlapping, aspects of the complex phenomena. Instead of immediately specifying a single joint model for all these data, it may be easier to instead specify distinct submodels for each source of data and then join the submodels together. We specifically consider chains of submodels, where submodels directly relate to their neighbours via common quantities which may be parameters or deterministic functions thereof. We propose chained Markov melding, an extension of Markov melding, a generic method to combine chains of submodels into a joint model.

When using any form of Markov melding, one challenge is that the prior for the common quantities can be implicit, and their marginal prior densities must be estimated. Specifically, when we have just two submodels, we show that error in this density estimate makes the two-stage Markov chain Monte Carlo sampler employed by Markov melding unstable and unreliable. We propose a robust two-stage algorithm that estimates the required prior marginal self-density ratios using weighted samples, dramatically improving accuracy in the tails of the distribution.

Expert information often pertains to the observable quantities in our model, or can be more easily elicited for these quantities. However, the appropriate informative prior that matches this information is not always obvious, particularly for complex models. Prior predictive checks and the Bayesian workflow are often undertaken to iteratively specify a prior that agrees with the elicited expert information, but for complex models it is difficult to manually adjust the prior to better align the prior predictive distribution and the elicited information. We propose a multi-objective global optimisation approach that aligns these quantities by adjusting the hyperparameters of the prior, thus "translating" the elicited information in an informative prior.