r/bigquery • • Oct 18 '23

Why is bigquery so expensive?

0 Upvotes

It’s like crack. It supercharges a startup - any idiot who can write sql can do incredible things.

They WHAM. Your 300 dollar bill is 30k. (Btw if any of you want to work out the math on how that happens, save it. You try bootstrapping a startup without a expert data engineer. Tech debt and. The deee credit con. If you are delivering product ou’ll be making fries not code.)

They are REALLY pushing it. I know it’s rough having your butt kicked off very by every azure and AWS. But stay the course. Add new features, keep prices low. We will be loyal. Raise your prices to make your investor targets - and people like me who spend high six low seven will leave.

You have been warned. It is a great product. Uniquely accessible to new employees.

You are KILLING a your fast growing companies before they have a chance.


r/bigquery • • Apr 11 '20

Created a Dashboard using Big Query COVID-19 public dataset and Data Studio Plans to use GeoViz instead of DataStudio.

Thumbnail
youtu.be
0 Upvotes

r/bigquery • • Jan 31 '25

What errors may I have made that I’m getting no data display

Post image
0 Upvotes

r/bigquery • • Oct 14 '21

Helping With This Error. I keep getting this error. I am lost on how to fix the error. I'm probably overthinking the solution. Any suggestions would be greatly appreciated.

Post image
0 Upvotes

r/bigquery • • Dec 19 '19

How to Analyse Facebook Ads using Google BigQuery Automation

Thumbnail
medium.com
0 Upvotes

r/bigquery • • Aug 04 '22

Infuriating arrogance from BigQuery support

0 Upvotes

I raised a bug with BigQuery support. It took months for them to get back to me. Then they clearly did not bother to read the bug report and gave me responses that are completely irrelevant, then closed the bug ticket. This is extremely frustrating. I have to question the culture of the BigQuery engineering team and the quality of product.

Original bug report in April:

Expected behaviour: "WEEK" as an acceptable date_part, as per documentation
https://cloud.google.com/bigquery/docs/reference/standard-sql/datetime_functions#datetime_diff

Actual behaviour: received this error "DATETIME_DIFF does not support the WEEK date part at [23:3]"

Example code:

datetime_diff(timestamp_millis(cast(events.createdAt as INT64)), timestamp_millis(cast(fv.startdate as INT64)), week)

(events.createdAt and fv.startdate are unix timestamps stored as numbers, hence had to be casted and converted first).

I have also tried "DATE_DIFF" instead of "DATETIME_DIFF" and experienced same error.

First response from Google, in June:

Since you are using the timestamp function, instead of using WEEK, you can use DAY as date_part. However, you will need to convert the day count into week count according to your needs.

You can try the below query for your requirement :

datetime_diff(timestamp_millis(cast(events.createdAt as INT64)), timestamp_millis(cast(fv.startdate as INT64)), day) // And then use the day count to calculate the weeks

I would recommend you to convert the timestamp to dates before, so that you can use the WEEK as date_part.

Please follow the above workaround, and let me know if it works for you or not?

This response ignores the fact that "WEEK" is listed in their documentation as an acceptable PART. I know the workarounds, that's not what I'm after. Besides, any business critical application would not be able to wait 2 months for an answer.

Second response from Google in August:
Status: Won't Fix (Intended Behavior)

Hello,

The issue is with using a date function with timestamp values.

Since date_diff
is a date function, your values must be dates.

For example:

select date_diff(date(current_timestamp()), date_add(date(current_timestamp()), interval -14 day), WEEK)

I will proceed to close this issue as Won’t Fix(Intended behavior).

However, if you still face any issue, please don’t hesitate to raise a new thread describing the issue.

My original bug ticket was raised against the DATETIME_DIFF function, not DATE_DIFF

They clearly did not bother to read the ticket. Such appallingly disrespectful behaviour.


r/bigquery • • Mar 09 '24

Saving $70k a month in BQ

Post image
0 Upvotes

Learn the simple yet powerful optimization techniques that helped me reduce BigQuery spend by $70,000 a month.

I think lot of folks can take help from this one: https://www.junaideffendi.com/p/how-i-saved-70k-a-month-in-bigquery

Let me know what else have you done to save $$$.

Thanks for reading :)


r/bigquery • • May 28 '23

Crear racha de usuarios

1 Upvotes

Estoy tratando de sacar la racha de los usuarios.

Si utilizan el servicio semanalmente de forma concecutiva. Si en una semana no consumen el servicio se reinicia el contador a cero. Es como la racha de duolingo.

Estoy usando rank() y over() pero no he podido. Alguno sabe?


r/bigquery • • Jun 22 '26

I built a CLI tool that analyzes BigQuery tables and explains what the data means using AI

1 Upvotes

Been a data engineer for 4 years. Every time I join

a new project, I waste hours understanding what

tables actually mean.

Built a CLI tool that analyzes BigQuery tables and

explains the business context using AI.

Demo: https://www.loom.com/share/af3409be37fa4692bb38b63b9f4a58cc

Happy to share the GitHub link in comments.


r/bigquery • • May 14 '25

BigQuery optimization? Don't migrate -- use this instead.

2 Upvotes

Hey folks, I'm launching a GCP big data processor and wanted to highlight my Hacker News launch here as well: https://news.ycombinator.com/item?id=43964505

tl;dr: ParaQuery is ~5x more efficient than BigQuery for many workloads, especially at scale -- without data migration, and with the ease of use that we've come to expect of BigQuery.

Let me know if such a tool would be useful to you!


r/bigquery • • Aug 11 '24

Data Analyst Copilot that works with BigQuery!

4 Upvotes

Hi all!

My name is John Bralich and I am the co-founder of a Miami based AI startup called the ai plugs (theaiplugs.com). We are working on a Data Analytics Copilot to help reduce time to insight and help you spend working on the stuff that matters most. We shared a demo yesterday with the help of one of our beta user companies. https://youtu.be/irNKDV29juQ?si=9orW0dnIJPSQAdSf. The demo is querying data stored in bigquery. Would love to hear your feedback and any other suggestions you have on features that would be beneficial to your everyday work!

Thanks,
John Bralich


r/bigquery • • Apr 13 '25

BQ FinOps ? Is it a thing ?

0 Upvotes

Hey all, I’m in an advanced stages of a really cool product that helps our team reducing our BQ cost in 50%+.

I wondered if it’s an issue in other teams as well, if so, what’s the cost of your BQ, is it storage mostly or processing ? and how you are able to reduce it ?

I’m really curious because I didn’t hear a lot of struggle around costs in BQ.


r/bigquery • • Aug 24 '22

Why don't you give data?

0 Upvotes

I have a query without errors, but it does not show me data, what could be the problem?


r/bigquery • • Jan 23 '22

Problem creating a table using CSV

0 Upvotes

I'm a beginner at BQ and have been struggling to create a table out of CSV. I'm using the console, added the Schema, then the error happens.

Failed to create table: Error while reading data, error message: Could not parse '2018/12/21' as DATE for field pg_start_dt (position 3) starting at location 0 with message 'Unable to parse'

For the date is there any rules that I need to follow? I couldn't find it anywhere.


r/bigquery • • Jan 07 '25

We are going to have a hybrid environment (on/off prem) for the next 5 years. What are the pain points coming?

0 Upvotes

There is going to be a lot of analytical queries and a lot of data blending between environments - we are not going to use virtualization to fix or cache anything.

I am talking terabytes of data here.

Besides latency (there is considerable distance between google and corp data center) what pain points await in this configuration?

Most data currently is on-prem, so im guessing data will flow from GCP to our internal datacenter.....


r/bigquery • • Feb 09 '23

export or download proyect from BigQuery

0 Upvotes

How can i export a project to Github, or is there any way to download it to my computer? I have been using BigQuery to practice SQL queries and i would like to export some of the things I've done to github in order to show my skills and find a job. any idea how i can do this? any help will be greatly appreciated. Im trying to build my portfolio as a data analyst.


r/bigquery • • May 23 '25

I've built a Cursor for data (Now working for BigQuery)

Thumbnail cipher42.ai
0 Upvotes

r/bigquery • • Jul 08 '20

I need some help with Gg Bigquery. This is my home work and i've never work with big query before.

Post image
0 Upvotes

r/bigquery • • 19d ago

Hiring - L1 Production Support Engineer

0 Upvotes

Job Code: TRX-26131
Company/Client: LTM
Experience: 4+ Years
Location: Pune / Hyderabad
Employment Type: Full-Time (FTE)
Budget: Up to ₹12 LPA
Shift: 24/7 rotational shifts, including night shifts (mandatory)

Role Overview
We are looking for an L1 Production Support Engineer with hands-on SQL querying skills, Python knowledge, and Google Cloud Platform (GCP) experience. The candidate will monitor production systems, troubleshoot data-related issues, investigate application failures, and support incident management processes.

Key Responsibilities
Monitor production applications and identify data-related issues.
Execute SQL queries to investigate, validate, and troubleshoot data discrepancies.
Support GCP services, including BigQuery, Bigtable, Pub/Sub, and Cloud SQL.
Use Python to assist with Dataflow troubleshooting and investigate Airflow failures and logs.
Understand ETL workflows and review Standard Operating Procedures (SOPs) for data-related applications.
Perform initial issue analysis, log investigation, incident tracking, and escalation.
Follow IT service management processes to ensure timely incident resolution.
Maintain incident records, operational documentation, and support procedures.

Required Skills & Qualifications
Mandatory hands-on SQL querying, including filtering, joins, LEFT JOIN with NULL handling, UNION, and analytical/problem-solving queries.
Practical experience with Python for troubleshooting and log analysis.
Working experience with GCP, particularly BigQuery, Bigtable, Pub/Sub, and Cloud SQL.
Ability to investigate Dataflow issues and understand Airflow failures and logs.
Basic understanding of ETL workflows and data processing pipelines.
Knowledge of incident management and IT service support processes.
Mandatory willingness and ability to work 24/7 rotational shifts, including night shifts.

Preferred Skills
Experience with ServiceNow or similar IT Service Management (ITSM) tools.
Experience supporting production data applications in a cloud environment.


r/bigquery • • Aug 09 '25

anyone else sees 50 big query ads on youtube every day?

1 Upvotes

is it standard spam marketing or did i do something wrong to be on the receiving end? its been months. and whats ironic is im willing to resign rather than use big query at this point if im ever asked to use it.


r/bigquery • • Aug 24 '26

Optimisation of bigquery for better performance cost for analytics

Thumbnail
github.com
3 Upvotes

r/bigquery • • Dec 08 '21

How to connect Google Ads with Big Query?

2 Upvotes

Hi there, i want to connect my Google ads account with Big query and pull the Advertising or Campaign Details from it. Can you advise me how to proceed on this?


r/bigquery • • Apr 12 '20

How to make a table size ZERO in BigQuery but still query it?

Thumbnail
youtube.com
2 Upvotes

r/bigquery • • Dec 31 '25

What do you use to query BigQuery outside of the Console?

Post image
0 Upvotes

The BigQuery web console is fine for quick queries, but when I'm doing deeper exploration or switching between multiple projects, it starts to feel clunky.

I've tried a few third-party clients but most seem optimized for traditional databases and BigQuery support feels like an afterthought.

What's everyone using? Bonus points if it handles BigQuery's nested/repeated fields well.

Also — I've been building a database client called Sheeta AI that recently added BigQuery support. Disclosure: I'm the founder. Would be interested to hear what features would make a BigQuery client actually worth switching for.