Simulated Annealing
One way to answer the problem of local maxima and related problems like ridges and plateaux in hill climbing is to permit the agent to go downhill to some extent. In virtual annealing named because of an analogy with cooling a liquid until it freezes. The agent picks to consider a random move. If the move improves the evaluation function, then it is constantly carried out. If the move doesn't improve the evaluation function, then the agent will carry out the move with some possibility between 0 and 1. The possibility decreases as the move gets bad in terms of the evaluation function, so really bad moves are hardly ever carried out. This strategy may often nudge a search out of a local maximum and the search can continue towards the global maximum.
Random Search
Some problems to be solved by a search agent are more creative in behaviour, i.e., writing poetry. In this type of case, it is often hard to project the word 'creative' on to a program because it is achievable to fully understand why it produced an artefact, by looking at its search path. In these type of cases, it is often a good idea to try some randomness in the search strategy, for an example randomly selecting an item from the plan to carry out, or assigning values from a heuristic measure randomly. This can add to the creative appeal of the agent, because it makes it much more difficult to predict what the agent will do.