To collect data from every single Calgarian would be prohibitive in terms of money, time, and resources. So, for all of The City’s telephone surveys, a smaller, random and representative sample of people are surveyed.

When respondents to a survey are selected at random from the population and reflect the population in terms of proportions of important variables such as age and gender, the survey results are said to be statistically representative of the population. For a telephone survey, a computer randomly selects phone numbers (both landline and cell) in Calgary. These are the people who are contacted and asked to take the survey.

One of the ways we ensure we hear from a sample of Calgarians who reflect the overall population in the city is by setting quotas for gender, age group, and quadrant of residence. Another way to ensure the survey sample is representative is using well-established data weighting methods during data analysis.

In the end, the results are representative of the city’s population within an acceptable margin of error. For a sample size of 500, the margin of error is ± 4.4% percentage points, 19 times out of 20. This means that if we did the survey 20 times with a random representative sample, results would be plus or minus 4%, 19 of those times.

If you’re interested, Statistics Canada is an excellent resource for helping understand surveys and representative sample.

Each year The City of Calgary conducts surveys with Calgarians to better understand their opinions and priorities regarding City services, policies, and plans. The results from these surveys are provided to City Council and Administration for consideration alongside other inputs so that best possible decisions are made on behalf of Calgarians.

For large city-wide surveys, The City of Calgary typically uses random digit dial telephone surveys (RDD). This approach randomly selects people to participate in telephone surveys, and offers the best population coverage because it includes a blend of cell and landline phone numbers. Participants are selected at random, and the survey is statistically representative of a population because everyone in the population has a known and an equal chance of getting selected to participate.

With online surveys, respondents are either part of an existing panel who have agreed to take part in multiple surveys over time or have responded to an open invitation to participate in a survey. Neither of these methodologies are considered to be representative of a population because in both cases the respondents are self-selected and not everyone has an equal chance of being selected to participate. This means that the results obtained may be subject to bias and skew and are not statistically representative of a population. For example, open link surveys posted on a website can sometimes be biased if individual respondents are able to respond to the survey multiple times.

The statistical reliability of the data collected in The City’s corporate level surveys, like the Citizen Satisfaction Survey is critical. For this reason the RDD telephone survey methodology is considered to be the best methodology.

The award of City of Calgary research contracts is subject to a competitive bid process. The City has a responsibility to ensure all public procurements are compliant with provincial and federal trade agreements.

These agreements prevent The City from awarding contracts based solely on the location of a bidding company. Although these agreements mean that The City of Calgary cannot limit City contracts to only local businesses, they also allow Calgary-based businesses to bid on contracts in other provinces.

Currently, there are three trade agreements that might be applicable to the procurements carried out by The City.

• The New West Partnership Trade Agreement (NWPTA) is shared between Alberta, Saskatchewan, Manitoba and British Columbia. The NWPTA stipulates that procurement values that meet the mandatory threshold of $75,000 for services (and goods) must be open for competition.
• The Canadian Free Trade Agreement (CTFA) is an intergovernmental agreement between the federal and each of the provincial and territorial governments. It specifies that each party is required to provide open, transparent and non-discriminatory access to procurement.
• The Comprehensive Economic and Trade Agreement (CETA) is a fair-trade agreement between Canada and Europe. It came into effect in September 2017. CETA encourages competition from the world’s second largest market and covers virtually all sectors and aspects of Canada-EU trade in order to eliminate or reduce barriers.