If a reader were to examine the vocabulary used in the novel, he
or she would be analyzing what?

The answer is that the reader would be analyzing the diction of a novel.

Explanation:

Diction refers to the linguistic choices a writer makes to effectively convey an idea, a point of view, or tell a story. The utilization of diction is the careful selection of words to communicate a message or establish a particular voice or writing style. For example, flowy, figurative language creates colorful prose, while a more formal vocabulary with concise and direct language can help a main point to come across as clear and precise.    

Writers use diction in order to choose specific words and phrases depending on the outcome they’re trying to achieve. Diction can:  

Create a certain tone that supports purpose. The purpose of a piece of writing determines its diction. In literature and fiction writing, writers often use informal diction and figures of speech—words used for non-literal meanings, like similes and metaphors. If a scientist is publishing a paper on their research, however, the language will be technical, concise, and formal, written for a specific audience. Support the setting. In fiction writing, the language an author uses supports the basic story elements, like setting. Diction helps establish when and where a story is set by using language native to that time and place, which is called colloquial diction. For example, a story set in New York City will have a different style of language compared to a story that takes place in London. Establish a narrative voice and tone. A writer’s attitude towards the subject of a story comes through in the words used throughout the story. This helps establish tone and impacts a reader's emotional response. For instance, the tone of a horror novel will be very different to that of a romance novel. Bring characters to life. A writer can tell a reader a lot about characters through their dialogue. The way a character uses diction reflects personal details like age and gender, background, social setting, and profession.

Diagram i think that’s right