The shift of technology start-up companies and talent into cities is taken for granted today, but I would never have predicted it even a decade ago.

The leading-edge high-tech companies of the 1970s, 1980s and 1990s, and even the early 2000s — like Intel, Apple and Google — were housed in corporate campuses in Silicon Valley. Microsoft had its headquarters in suburban Redmond, Washington; other high-tech companies clustered along the Route 128 suburbs outside of Boston, in the suburbs of Austin, Texas, or the office parks of North Carolina’s Research Triangle.

Back in the late 1980s, when I conducted my early studies of the geography of venture capital and high-tech industry with Martin Kenney, the lion’s share of venture capital backed start-ups were in these suburban areas.

That geography has become much more urban. With nearly $6.5 billion in such investment, San Francisco topped the $4.2 billion that the San Jose metro (which encompasses Silicon Valley) received in 2012, ranking as the world’s number-one location for venture capital investment that year. Greater New York took in more than $2 billion, the bulk of it in Lower Manhattan.

Those figures grew even larger in 2013. In that year, the San Francisco metro took in a whopping $8.5 billion in venture capital investment, with $6.2 billion of that flowing into the city itself, compared with $4.8 billion in the San Jose metro. Venture capital investment in Greater New York grew to more than $3 billion.

Across the US, more than half of this investment and nearly six in 10 start-ups were in urban ZIP codes in 2013. Roughly 60 per cent of venture investment in the Bay Area went to dense, walkable neighbourhoods in urban ZIP codes, and in New York, more than 80 per cent did.

In the ZIP codes that received venture capital investment nationwide, the share of workers who walked, biked or used transit to get to work was nearly twice the national average (16.6 per cent in venture capital neighbourhoods, versus 8.4 per cent overall). More than a quarter of venture capital investment nationally was concentrated in neighbourhoods where more than half of all workers walked, biked or used transit, and more than a third was located in neighbourhoods where more than 30 per cent did so.

Indeed, urban density has become one of the most important factors in the propagation of high-tech start-ups and the attraction of venture capital investment. The amount of venture capital invested in start-ups is more strongly correlated with population density than it is with either the concentration of highly educated people or the concentration of the creative class, and only slightly less strongly correlated with population density than it is with the concentration of high-tech industry — which is the very thing that attracts venture capital investment to begin with.

Even when such investment flows to start-ups in smaller cities and suburbs, it tends to go to the ones that have the most urban characteristics. With more than $1.5 billion in investment, the leading centre for venture capital investment in Silicon Valley is the densest part of Palo Alto surrounding Stanford University. Cambridge, Massachusetts, home to the Massachusetts Institute of Technology and Harvard, attracted close to $1 billion, more than the suburbs out along Route 128.

Dense, walkable Santa Monica attracted double the amount of venture capital that the much larger but more sprawling city of Los Angeles received.

The urban shift in technology start-ups extends beyond the US. Once a high-finance centre with little in the way of tech, London is now home to thousands of high-tech companies and tens of thousands of jobs. Its start-ups pull in more than $800 million a year in venture capital, more than in Seattle or Austin.

Beijing and Shanghai, Mumbai and Bengaluru, Toronto, Paris and Moscow all number among the world’s 20 leading locations for venture capital investment and start-up companies. Urban start-up hubs are also emerging in Berlin, Amsterdam, Liverpool and Munich in Europe, and Tel Aviv and Amman, Jordan, in the Middle East.

Start-ups and cities are a natural match. Urban areas provide the diversity, creative energy, cultural richness, vibrant street life and openness to new ideas that start-up founders are looking for.

While many large, well-established tech companies require large headquarter sites and remain in the suburbs, start-ups can make use of the adaptable work spaces that urban industrial and warehouse buildings provide.

In the past, the most successful start-ups were those that focused on developing and manufacturing software or hardware, and the large facilities and campuses they required could be more cheaply accommodated in the suburbs.

Today’s hottest start-ups involve digital and social media, games and creative applications, which draw on the deep pools of designers, composers, scenarists, musicians, marketers and copywriters that can be found in cities.

Other companies, such as Uber and Airbnb Inc., were set up to actually make some aspect of urban life work more efficiently — transportation and short-term housing, respectively. Cities aren’t just locations for these companies, but the sites of the very problems their technologies aim to solve and the platforms for innovation itself.

Indeed, the cultural creativity of great cities has proved to be a big draw for talent. New York venture capitalist Fred Wilson told me of a meeting he had with Etsy Inc.’s founder, Rob Kalen. Wilson found him sitting in his office playing his guitar. When Wilson remarked that he hadn’t realised what a talented musician he was, Kalen said: “Fred, you know I’m really an artist. If I had grown up in the 1960s, I would have been a folk musician. If I’d grown up in the 1920s, I would have been a painter. But I grew up in this generation, and my art is making websites.”

Wilson added, “There’s more art than science in this kind of tech, and artists, for the most part, congregate in cities.”

Back in 2002, in “The Rise of the Creative Class”, I noted the connection of bohemian values, vibrant art and music scenes, and the places that incubate high-tech start-ups. San Francisco was a font of 1960s psychedelia and the home of bands like the Grateful Dead, Jefferson Airplane and Big Brother and the Holding Company.

Seattle was the birthplace of Jimi Hendrix, who had such a powerful influence on Microsoft co-founder Paul Allen that Allen opened the Experience Music Project downtown; the city was also the birthplace of the grunge scene that later produced Nirvana. Austin, too, has a vibrant alternative music scene that grew up alongside its tech scene. New York and London are among the world’s artistic and cultural leaders.

These are not mere coincidences: Great cities are creative and innovative across the board. In fact, my research shows empirically that artistic and cultural creativity acts alongside the high-tech industry and business and finance to power economic growth.

The trend is not so much a startling reversal as a correction of a historical aberration. The venture capital icon Paul Graham saw the writing on the wall in 2006. For all its advantages and power, he wrote, Silicon Valley had a great weakness. This high-tech “paradise”, with its roots in the 1950s and 1960s, had become “one giant parking lot,” he observed.

San Francisco and Berkeley are great, but they’re 40 miles away. Silicon Valley proper is soul-crushing suburban sprawl. It has fabulous weather, which makes it significantly better than the soul-crushing sprawl of most other American cities.

But a competitor that managed to avoid sprawl would have real leverage. He was right — that’s exactly what happened.