The ramblings of Steve-0Posts RSS Comments RSS

Archive for the Tag 'mysql'

PostgreSQL: Debugging “missing chunk number X for toast value Y” errors

If you found this through a search, you probably have seen something like this in your logs:
ERROR: missing chunk number 0 for toast value 87246446

Searching for this issue on the net, I found that this is usually caused by a corrupt row in the database. The solution is to find the corrupt row, and delete it (and/or update it with proper, non-corrupt values).

Unfortunately, there is no easy way to find the row that is giving you grief (or even the table if you don’t know what is causing this error). You basically have to narrow down your search by looking at parts of the database. If you have some sort of date or timestamp column in your table, you can narrow it down a lot more easily, if you know approximately when the error started:


mydb=# select id from mytable where date_created > timestamp '2009-02-27 00:00:00' and date_created < timestamp '2009-03-01 23:59:00';
id
--------------------------------------
3ab226cf-a972-463d-b5a1-148fe39672b5
10daca73-b2b3-470c-a258-5c92d21cfbb6
(2 rows)

Luckily it's only two rows, and the first one is the culprit:

mydb=# select * from mytable where id='3ab226cf-a972-463d-b5a1-148fe39672b5';
ERROR: missing chunk number 0 for toast value 87246446

Now, on to delete it:
mydb=# delete from mytable where id='3ab226cf-a972-463d-b5a1-148fe39672b5';

If you don’t have a date/timestamp column in your table, you could use LIMIT and OFFSET to narrow it down. I.e. a series of statements like (increasing the offset each time):

select * from mytable order by id limit 1000 offset 5000;

No responses yet

Creating a true read-only user in PostgreSQL

I develop a product at work that (among many other things) allows users to easily create and manage databases to interact with other packages on their appliance. One feature that was requested and green-lighted is the ability to create a read-only user for existing databases- a user that can connect to a given database and access all tables and views in that DB, and nothing else.

Since PostgreSQL is our database of choice, I started researching the process, thinking it would be a couple mysql-esqe GRANT statements and that would be it. Turns out that it is a huge PITA in postgres – even my solution I’m documenting here has its shortcomings, but as far as I know is the best / only way to accomplish this task. I ran across quite a few sites that helped with pieces of this, but none that actually tied the process together for production usage. I am by no means a Postgres expert, but do have a good bit of experience mucking around with back-end settings and figuring how to script common tasks.

There are two main things to watch out for when trying to create a read-only user in PostgreSQL, especially if you come from a DB like MySQL:

  • PostgreSQL only sets permissions on objects, not on databases, so you need to grant read access to all your tables/views/etc, and if you add a table down the line, you need to remember to manully grant read access to it after creating the table.
  • I’m guessing 95% of postgres users just use the default “public” schema, and as such, you need to revoke create privileges from the PUBLIC group. Otherwise, your “read-only” user will still be allowed to create tables that it owns, even if you’ve only given it read only access to all other objects in your database.
  • For this example, we’ll use database name “mydb”, database user/owner “mydbuser”, and we’ll create a read-only user named “mydbuser_ro”. This assumes that you did not define a schema for your database and are using the default “public” schema.

  • Revoke default permissions from public group:
    REVOKE CREATE ON SCHEMA public FROM PUBLIC;
    REVOKE USAGE ON SCHEMA public FROM PUBLIC;
  • Add back permissions for your database owner:
    GRANT CREATE ON SCHEMA public TO mydbuser;
    GRANT USAGE ON SCHEMA public TO mydbuser;
  • Create the new user via the command line, or pgadmin/etc:
    psql -U postgres -t -c "create role mydbuser_ro password 'abc123' NOSUPERUSER NOCREATEDB NOCREATEROLE INHERIT LOGIN;
  • Grant usage permissions for your read-only user:
    GRANT USAGE ON SCHEMA public TO mydbuser_ro;
  • Grant select permissions on all database tables from the command line:
    psql -U postgres -qAt -c "select 'grant select on ' || tablename || ' to \"mydbuser_ro\";' from pg_tables where schemaname = 'public'" mydb | psql -U postgres mydb
  • Setup remote access for the read only user in pg_hba.conf as appropriate
  • Once complete, you can verify the settings with a quick sql query. You should see something like this, with the user=UC (for Usage/Create), and user_ro=U (for Usage):
    mydb=> select * from pg_namespace where nspname='public';
    nspname | nspowner | nspacl
    ---------+----------+------------------------------------------------------------------
    public | 10 |
    {postgres=UC/postgres,mydbuser=UC/postgres,mydbuser_ro=U/postgres}

    One response so far