Most the policy recommendations are to the benefit of insiders and regulators, not citizens and residents

From Your questions, answered by Timothy B. Lee.  There are at least three significant urban issues roiling public debate today, mostly centered in large dysfunctional blue cities but appearing in various forms elsewhere as well.

1)  Gentrification

2)  Expensive housing

3)  Densification 

Gentrification is almost always at least a class issue and sometimes a manufactured race issue and is frequently popular on the Left.  If a city becomes popular as a residential destination, and as long as there is already significant land use, then the only option for adding additional residents is by knocking down poor neighborhoods and building housing for wealthier people wanting to move in.  

That is why it is always a class issue.  Even though it is rarely discussed on a class basis.  It is much more frequently cast as a race issue because poverty and race are frequently correlated.  If you are building in a poorer neighborhood, on average, you are building in a blacker neighborhood.

I used to be much more sympathetic to the anti-gentrification argument, more for emotional reasons than necessarily logic.  I accepted the representation that these poorer neighborhoods were being systematically dismantled to the disadvantage of long term residents.  And I remain sympathetic to that aspect.

The problem is that it appears that that aspect just isn't as prevalent as is assumed.  Five, maybe ten years ago I came across three studies, all with the similar findings.

Poor neighborhoods are dominated by rental housing rather than more settled and stable ownership housing.  This was combined with very high churn.  I don't recall the specific numbers but something like 20%, maybe more, of residents moved in and out of the neighborhood each year.  At that rate, a neighborhood would turnover completely within five years.  

These are averages of course.  There would certainly be some home owners who are very settled who would be affected by the changing nature of the neighborhood.  Just not many.  

The key finding was that the argument against gentrification was largely a mom and apple pie one of not destroying settled neighborhoods.  But the reality was that there was no there there.  These were no settled neighborhoods.  Just transience and churn.

Gentrification seems far more benign to me now than a few years ago.  There is a residual issue that older, long term residents who are home owners can be forced out of their homes owing to inflating land values on which they can't afford the rising taxes but that is a separate issue that is amenable to some solutions.

Right now, gentrification opposition seems mostly virtue signaling.

Expensive Housing is also a hot topic.  Since cheap housing is not a positive right in most jurisdictions, this seems to me to be primarily a partisan signifier and again a mechanism of virtue signaling.  

Housing, with a few wrinkles, is much like any other commodity.  The primary wrinkle is that of housing as a positional good.  

Positional goods are goods valued only by how they are distributed among the population, not by how many of them there are available in total (as would be the case with other consumer goods). The source of greater worth of positional goods is their desirability as a status symbol, which usually results in them greatly exceeding the value of comparable goods.

Various goods have been described as positional in a given capitalist society, such as gold, real estate, diamonds and luxury goods. Generally any coveted goods, which may be in abundance, that are considered valuable or desirable in order to display or change one's social status when possessed by relatively few in a given community may be described as positional goods. What could be considered a positional good can vary widely depending on cultural or subcultural norms.

The policy solution to expensive housing is conceptually simple in a market economy.  Allow more building.  More housing per se or greater densification, it doesn't matter.  More supply brings down costs.  In theory.

In practice?  Well, despite rhetoric, what is true in theory is true in practice.  Lee links to two papers:  Does Building New Housing Cause Displacement?: The Supply and Demand Effects of Construction in San Francisco by Kate Pennington and Supply Shock Versus Demand Shock: The Local Effects of New Housing in Low-Income Areas by Brian J. Asquith, Evan Mast, and Davin Reed.

In both cases, they find that additional housing supply reduces the cost of housing (in terms of rents).  

From the Abstract of the San Francisco paper.

Major cities around the world are gentrifying as high-income newcomers drive up housing prices and displace lower-income incumbent residents. Increasing the housing supply should mitigate rent increases, but new buildings could also stimulate demand for nearby housing by improving neighborhood quality. The net impact depends on how the relative sizes of these supply and demand effects play out over space. This paper identifies the causal impact of new construction on nearby rents, displacement, and gentrification in San Francisco by exploiting random variation in the location of new construction induced by serious building fires. I combine parcel-level data on fires and new construction with an original dataset of historic Craigslist rents and a panel of individual migration histories to test the impact of proximity to new construction. I find that rents fall by 2% for parcels within 100m of new construction. Renters' risk of displacement to a lower-income neighborhood falls by 17%. Both effects decay linearly to zero within 1.5km. Next, I identify a hyperlocal demand effect, with building renovations and business turnover spiking within 100m. Gentrification follows the pattern of this demand effect: parcels within 100m of new construction are 2.5 percentage points (29.5%) more likely to experience a net increase in richer residents. Affordable housing and endogenously located construction do not affect rents, displacement, or gentrification. These findings suggest that increasing the supply of market rate housing has beneficial spillover effects for incumbent renters, reducing rents and displacement pressure while improving neighborhood quality.

From the Abstract of the Supply Shock paper.

We study the local effects of new market-rate housing in low-income areas using microdata on large apartment buildings, rents, and migration. New buildings decrease nearby rents by 5 to 7 percent relative to locations slightly farther away or developed later, and they increase in-migration from low-income areas. Results are driven by a large supply effect—we show that new buildings absorb many high income households—that overwhelms any offsetting endogenous amenity effect. The latter may be small because most new buildings go into already-changing areas. Contrary to common concerns, new buildings slow local rent increases rather than initiate or accelerate them. 

Both of them support that increasing supply reduces price.  As you would expect.  

Which is interesting because the common concern of all classes of home owners is with regard to any policy or action which might reduce the value of their own property.  The usual counter argument is the claim that higher end real estate development leads to higher property values.  These papers are measuring in terms of rental rates rather than home valuations so maybe there is a different dynamic going on, but it appears that increasing building reduces rental prices and therefore probably reduces home valuations.  

Densification is a common policy urged by urban planners and property developers.  The former for theoretical and ideological reasons and the latter for pragmatic commercial interests.  Densification takes the form of increasing the number of multiunit residential dwellings (apartment buildings) or increasing the number of accessory dwelling units (ADUs).  In theory ADUs are mother-in-law suite coach houses on the same property.  It allows more people to live in a given area without a complete rebuilding of the area.

There is a definite ideological and progressive element to densification policies, often with a vibe of getting back at the rich by building apartment buildings in single family neighborhoods and increasing the amount of subsidized housing or the number of set asides for lower income rentals.

It is a price of admission for developers which they can absorb through increasing access to high value areas.  It is a feel-good progressive jolt to be seen to be robin hooding for the poor, even though, most often such set asides only actually yield bribes from developers to government officials or access to those subsidized units in nice neighborhoods going to those well connected at city hall, often family.  

What is rarely discussed, or at least I have not seen it treated, is that changing zoning rules in order to increase densification, whether to better integrate neighborhoods or to reduce housing costs, is a form of taking.  Zoning is usually a long term commitment between government, residents and developers, allowing all to make reasonably informed and optimizing decisions (where to build, what kind of quality of stock to build, investments to be made, etc.)  

As long as everyone adheres to the same rules over time, it usually results in increasing value for all property owning residents.  If you change the rules, it usually effectively results in a transfer of value from home owning residents to developers and to city bureaucrats.  It is an indirect taking or a tax increase if you will.

I would be interested to see research on this.  In the meantime, in practical terms, all calls for increasing densification with which I am familiar are clearly attempts to transfer power and money from established property owners to property developers and their city hall allies.  

My net position is 1) Don't worry about gentrification except when there are very particular conditions which are pretty rare.  2) Don't worry about densification.  It will happen or not.  If demand does go up, then let increasing demand pull in development within the existing zoning constraints.  Do not change zoning as it represents a major loss of trust in government affecting long term development.  3) Allowing more building, within the existing zoning constraints, will also usually reduce housing costs.  

UPDATE:  From In Philly, There's No Such Thing as Free Affordable Housing by M. Nolan Gray.

In late January, Philadelphia mayor Jim Kenney signed into law a city council bill mandating that newly constructed apartment buildings in the Third and Seventh Council Districts – University City and parts of North Philadelphia, respectively – must include income-restricted housing. According to the new rules, which take effect in six months, any multifamily building with 10 or more units must cap rents in 20% of units at a rate affordable to households earning 40% of the area median income or less. 

It’s a well-meaning policy, but in practice, it will likely worsen housing affordability in Philadelphia.

This isn’t the first time that the city has attempted an “inclusionary zoning” policy. In recent years, Philadelphia offered developers additional height or floor area in certain pockets of the city in exchange for including income-restricted housing. But these earlier programs were voluntary. Developers, moreover, could pay into the Philadelphia Housing Trust Fund in lieu of setting aside income-restricted units.

While few developers opted to set aside actual units – building and maintaining income-restricted units can be costly and complex – many were more than happy to make such contributions. And so, today, the Housing Trust Fund is now flush with cash to a level that city leadership is quietly diverting it to other uses.

The recently adopted ordinance changes this policy, making participation mandatory and blocking developers from simply paying money to meet their obligations. There’s no more opting out if the mandates don’t pencil. 

Read the whole thing.  If you want cheaper housing, don't pursue it through indirect regulation.  Make the market work better and more transparently.