A challenging problem facing the semantic search of multimedia data objects is the ability to index them. Here, we present an architectural paradigm for collaborative semantic indexing, which makes use of a dynamic evolutionary approach. By capturing, analyzing and interpreting user response and query behavior, the patterns of searching and finding multimedia data objects may be established. Within the present architectural paradigm, the semantic index may be dynamically constructed, validated, and built-up, where the performance of the system will increase as time progresses. Our system also incorporates a high degree of robustness and fault-tolerance whereby inappropriate index terms will be gradually eliminated from the index, while appropriate ones will be reinforced. We also incorporate genetic variations into the design to allow objects which may otherwise be hidden to be discovered. Experimental results indicate that the present approach is able to confer significant performance benefits in the semantic searching and discovery of a wide variety of multimedia data objects.