Personalized Nutrition-Aware Dietary Recommendation With Multimodal Large Language Models

Poor diet quality is a major modifiable risk factor for obesity, cardiovascular disease, and metabolic disorders, motivating a growing ecosystem of diet-tracking apps and recipe platforms that promise personalized guidance. However, most of these systems still rely on simple scoring rules or ID-based collaborative filtering that ignore rich multimodal food information (ingredients, cooking methods, images, nutrient profiles) and lack explicit nutritional reasoning. To address this gap, we propose NutriRec, a multimodal LLM-based framework for personalized dietary recommendation that links nutritional reasoning with collaborative dietary patterns. We uses a multimodal LLM to generate unified representations of meals and food items from textual descriptions, ingredient lists, nutrient profiles, and images, capturing both sensory and nutritional attributes. On top of these content-based embeddings, NutriRec incorporates collaborative filtering signals through a post-alignment mechanism that aligns user representations from an ID-based sequential model with those from the multimodal LLM. This design enables explainable dietary reasoning by grounding each recommendation in concrete nutritional evidence (e.g., lower sodium, higher fiber, heart-healthy tags) while still reflecting behavioral patterns learned from large-scale consumption logs. We evaluate NutriRec on realistic dietary recommendation scenarios with long-term, multimodal food interaction histories. Across these settings, NutriRec not only yields stronger recommendations than sequential and LLM-based baselines, but also provides more explainable dietary reasoning by explicitly linking suggestions to nutrient profiles, meal composition, and simple guideline-based constraints.