Housing Need and Data Accuracy: Getting the Numbers Right

The urgency to address housing provision and its role in exacerbating inequality has only increased following the July 2024 election. The demand for more homes, a concern not unique to the UK, raises key questions: How many homes are needed? Where should they be built? For whom? And how long will they serve their purpose?

The inspiration for this article stems from a question I've pondered for years: How many homes or flats in the UK are sitting vacant? This thought has intrigued me, especially as I’ve lived in a high- density development in central London where, over the past 16 years, occupancy has rarely exceeded 50%. The question first struck me during a conversation with an electrician from the Berkeley Group, one of the UK’s leading housebuilders, who mentioned that 75% of the surrounding properties remained vacant for most of the year. This observation has stayed with me, especially as I've volunteered at a homeless shelter for over 16 years, where discussions about housing shortages have only grown more urgent. Often, on my way home, I wonder about the empty homes opposite mine, many of which sit vacant year after year or are only sporadically occupied. Do we really have a housing shortage, or is the issue more complex than it seems?

Getting the stats right:

According to the Policy Exchange, during much of the post-war period, the UK's housing stock grew faster than its population. By the 2021 Census, long-term vacant dwellings in England accounted for 1.3% of the stock, while second homes represented 2.8%, based on data from the Ministry of Housing, Communities and Local Government (MHCLG). The Redfern Review (2016) reported that housing stock grew 14% faster than the number of households, with dwellings exceeding household formation by 70% between March 1996 and 2018. Over the same period, the population increased by 14.6%, while residential property floor area expanded by 25.7%. Whether this increase in housing stock has affected prices remains a matter of debate. However, it’s clear that declining real incomes are outpacing both rent and land values. For most people, regardless of generation, saving for a home deposit or affording the 32-50% of their income going to rent has become increasingly difficult, contributing to the rise of "hidden households."

The shortage of affordable housing in desirable locations, combined with a lax approach to regulating landlords, has led to overcrowded living situations. Properties designed for a couple may house five, or families of four may be crammed into one-bedroom flats. The growing number of unaccounted residents has introduced new phenomena such as hidden households, "sofa surfers," adults living at home longer, overcrowding, and unlicensed conversions serving as makeshift housing. All of this complicates the ability to draw firm conclusions about the housing market. As Lindsay Judge's data suggests, there are significant gaps in measuring housing supply and demand, especially when a large proportion of developments consist of single-bedroom flats. As Judge notes, “there is no official estimate of family units in the UK," making it difficult to conclude whether supply and demand are well aligned (Judge, 2019).

Housing provision is influenced by several factors that often disregard local needs. A significant part of the problem lies in the diverse nature of housing markets, defined by tenure, property type, prices, locations, and demographics. Central to housing policy is the idea of a "housing ladder." However, over the past three decades, housing has increasingly been viewed as an investment, with Buy-to-Let properties and second homes prioritized for capital gains rather than fulfilling the UN's principle of equal access to housing as a human right. Consequently, the housing ladder concept has largely vanished, along with the affordability and diversity of available homes. This has led to a widespread belief that there is a housing shortage—but is that really the case?

Across the UK, there may be enough housing to meet everyone's needs. The real issue lies in the unequal distribution of this accommodation, as housing has increasingly become a commodity, losing its true function and value. The 2015 English Housing Survey revealed that while 54 million people lived in households, they only required 37 million bedrooms to be adequately housed according to the bedroom standard. However, 63 million bedrooms were available nationwide, leaving 9 million unused.

Currently, 53% of owner-occupied households under-occupy their homes, compared to just 10% of social rented households. Demographic trends show that household formation has been constrained, suggesting that new housing supply has been too low to meet both housing aspirations and needs. Although building new homes may not dramatically improve affordability or increase homeownership, it remains a crucial part of the solution imposed on local authorities, regardless of the availability of existing housing.

The existing housing supply sees only 1.2 million homes change hands each year, and this low turnover restricts labour mobility, negatively impacting economic activity—especially within the housing ladder model most policies are based on. Traditionally, households move from flats to family homes, and then downsize later in life. However, the low turnover of existing stock has become a significant barrier to a functioning housing market, yet policymakers have shown little interest in addressing it.

The growth of landlords and the private rented sector has been fuelled by the introduction of assured shorthold tenancies and the Buy-to-Let mortgage market. This has led prospective first-time buyers to compete with investors, who often outbid them. Affordable housing policies vary widely, based on factors like the number of proposed dwellings, the size of the site, or the development's floor area. Local authorities may have different thresholds in different housing zones within the same borough, leading to contradictions and confusion across neighbourhood plans and core strategies.

There are no standardized affordable housing requirements nationwide. According to the 2023 National Planning Policy Framework (NPPF), paragraph 66 mandates that at least 10% of homes in any development should be affordable for homeownership. However, planning policies lack clear standards, making them open to interpretation, and 67% of these policies are estimated to be outdated. Accurate data is crucial for effective policymaking, as it can close gaps in implementation and improve consistency and confidence.

Measuring housing supply accurately remains a challenge due to limitations in the available data. It’s difficult to determine how many new homes are being built, where they are being constructed, or by whom. This is a significant issue. Data from MHCLG, which feeds into the Office for National Statistics' dwelling stock by tenure dataset, relies heavily on the National House Building Council (NHBC). It was once believed NHBC accounted for 85% of new builds, but research suggests that fewer homeowners now use NHBC’s services due to pricing issues, leading to distortions and inaccuracies in MHCLG data—potentially undercounting homes by at least 30,000 annually since the 1990s.

The National Housing Federation also publishes data on housing delivery by housing associations, but this data often overlaps with other sources, contributing to the overall inaccuracy of new build figures. While Energy Performance Certificates (EPCs) and council tax records offer some insights, they still do not provide a complete picture for assessing local housing needs and usage.

Local markets face various affordability challenges. Housing prices are measured through five main indices: the UK House Price Index (UK HPI), LSL Acadata HPI, Halifax HPI, Nationwide HPI, and Rightmove HPI. These indices use data from sources such as the Land Registry, mortgage lending, property listings, and transaction numbers, adjusted through various methods. The Department for Levelling Up, Housing and Communities’ 2015 program aimed to build over 160,000 homes through its land disposal initiative. However, the government did not track how many homes were actually completed on that land.

Localise the data management and collection.

In 2001, the government launched an initiative to assist key public sector workers in renting or buying homes through housing associations, aiming to address retention and recruitment issues in essential services. This built upon earlier schemes, such as the 1920-1994 Metropolitan Police housing project, which was eventually sold off. Initially introduced as the Starter Homes Initiative under Tony Blair's government, it later evolved into the Key Worker Living scheme, helping key workers climb the housing ladder. Registered Social Landlords were granted funds to build homes with intermediate rents or discounted ownership, which influenced broader policies like Help to Buy and shared ownership.

Revitalizing a Key Worker Housing policy can greatly benefit society by allowing essential workers— such as healthcare professionals and police officers—to live closer to work, thus reducing the risks associated with long commutes. A key lesson from the COVID-19 pandemic is the critical role frontline staff play and the unacceptable housing conditions many are forced to endure. Rising costs for private developers and post-pandemic office vacancies present opportunities to prioritize key worker housing on public land, possibly engaging smaller builders. The Canadian government’s approach, leasing public lands to developers at reduced costs through long-term leases, offers a model that could be adapted. This is similar to the Public Land for Housing Programme proposed by the Policy Exchange (Vitali, 2024).

The government aims to incentivise developers to prioritise affordable housing through what is known as the leasehold infrastructure delivery model. This approach, especially useful for large-scale housing and development projects, allows developers to save on outright land purchases, freeing resources to focus on construction. This would enable the building of more units at a faster pace, using higher-quality materials and sustainable design features. A Public Lands Acquisition Fund could be established to purchase land from government entities, making it available for housing developments where they are needed, for those who need them. Exploring the use of Real Estate Investment Trusts (REITs) to fund key worker housing is another potential economic advantage. By leasing local public land for housing essential workers, local authorities can establish a framework with specific criteria to prioritize the local workforce. Involving local builders and maintaining detailed records of existing and proposed housing stock ensures the framework is responsive to local needs, rather than market demands, thus preventing vacant or overcrowded homes and addressing homelessness more effectively.

Conclusion

The built environment, particularly housing, is often at the heart of societal change. Today, the real estate market is pivotal in rethinking global capital strategies in response to emerging technologies like AI and robotics, the impact of COVID-19 on workforce structures, and the shift to hybrid working models. Additionally, it must address the challenge of designing resilient products to withstand climate volatility. Amid these transformations, housing affordability has become increasingly difficult to navigate.

Innovations in data analysis hold the potential to clarify the housing sector by efficiently processing large datasets and eliminating redundancies, thereby mapping out housing needs and market trends. However, focusing solely on building homes for sale can undermine essential services and community functionality. True progress requires identifying underutilized spaces in key areas and repurposing vacant properties for permanent, high-quality housing.

Accurate data across the health and housing sectors is crucial, as it directly influences population well-being and life chances. Although the task may seem daunting, the first step is to get the numbers right. We need a clear understanding of who lives where and in what conditions, ensuring that every decision is built on a foundation of existing assets. Housing is the most expensive and impactful asset in a person's life, and it must be properly accounted for.

In 2012, the Health and Social Care Act was adopted, transferring Public Health responsibilities from the NHS back to local authorities. This move reinforced local governments' duty to improve local health and coordinate efforts to protect the public. UK local authorities possess extensive local information that is publicly scrutinized and regularly updated, providing essential insights into assets, geography, health profiles, and future development plans, including the decentralization of energy.

Housing is intrinsically a health issue, which likely explains its placement under the Ministry of Health in the 1930s. Effective housing delivery requires clear direction and ownership by organizations with a proven history of prioritizing the local population’s best interests. These organizations must leverage accurate data to determine the appropriate types, tenure, and access to housing as needed by their communities.

Suggestions Going Forward:

1. Utilize AI platforms currently employed in the real estate sector to create a comprehensive record of the nation's housing supply. This should include where properties are located, who occupies them, and whether they can be better utilized or improved for more efficient, long- term use.

2. Incentivize local authorities to reclaim their role in housing their populations by linking local health and productivity improvements to measurable social value, which can attract investor interest.

3. Expand government permission allowing councils to levy double council tax on long-term empty properties to include "buy-to-leave" homes. Additionally, introduce a local tax band for properties that remain unoccupied for 75% of the time.

4. Tax private land left undeveloped in regions with homelessness at a punitive rate for every month the land remains unused.

5. Instead of selling off public land, lease it to small and medium-sized house builders for development, fostering local growth.

6. Prioritise housing for essential workers and the critical foundations of society to balance productivity with sustainable community structures and protect local economies.

References:

1. Andersen B, Eline Ander H, Skrede J. The directors of urban transformation: The case of Oslo. Local Economy: The Journal of the Local Economy Policy Unit. 2020 Nov;35(7):695–713.

2. https://policyexchange.org.uk/wp-content/uploads/2019/11/Revitalising-Key-Worker- Housing.pdf

3. Property Science Seminar – Neal Hudson {Internet}. Vimeo. 2024 {Cited 2024 August 20} Available from https://vimeo.com/user63815236/review/302788568/826ff40810

4. Bootle R, Vitali J. The UK’s Broken Housing Market Causes, Consequences and Cures The Second Part of Policy Exchanges Policy Program for Prosperity [Internet] Available from: https://policyexchange.org.uk/wp-content/uploads/v3-PX4-The-UKs-Broken-Housing- Market-Web.pdf

5. Redfern Review into the decline of home ownership [Internet]. Available from: https://thinkhouse.org.uk/site/assets/files/1841/redfern.pdf

6. The one million missing homes? • Resolution Foundation [Internet]. Resolutionfoundation.org. 2019 [cited 2024 Aug 21]. Available from: https://www.resolutionfoundation.org/comment/the-one-million-missing-homes/