FYI
Jennifer Cline, P.E.
City of Molalla
Public Works Director
O: 503.759.0218
F: 503.829.3676
"From: Troy Collison [mailto:tbc@industrialsystems-inc.com]
Sent: Tuesday, May 12, 2015 8:27 PM
To: Jennifer Cline
Cc: Dan Huff; Jason Clifford; Matt Riedl; Mark Lietzke; John Sowards
Subject: RE: WWTP - TPS high level alarm
Hi Jennifer,
Today, we put a significant amount of effort into the creation of the aforementioned proposal.
It seems the deeper we dig into the state of the Ethernet network that supports the control system at this site, the more we discover that needs to be addressed as a part of this undertaking.
To make a long story short, there are some cabling issues at the WWTP, a sonic wall (firewall device) that has not be updated in a decade or more, and there appears to be a City installed Ethernet switch at the site that interconnects with the control system Ethernet network, but we are not sure how and/or why. This latter device also seems to be connected to the City’s phone system. All these things are pretty simple to sleuth out and document so we can do that, but we need to take the extra time in preparing our proposal to do our best to ensure the final scope provides the correct solution to this situation.
I apologize for the fact this process is taking longer than I thought it would.
We will have a proposal to you tomorrow before noon.
And yes, absolutely, I will be available to talk tomorrow morning. I will be in the office, but please call whichever number your prefer, they are all listed below in my email signature.
Best Regards,
Troy B. Collison
5835 NE 122nd Avenue, Suite #110
Portland, Oregon 97230
http://www.industrialsystems-inc.com
Phone: 503.262.0367
Cell: 360.957.5039
Fax: 503.262.0371
CCB # 196597 WA #INDUSSI880K9
AK# 1018436
From: Jennifer Cline [mailto:jcline@cityofmolalla.com]
Sent: Tuesday, May 12, 2015 5:36 PM
To: Troy Collison
Cc: Dan Huff; Jason Clifford
Subject: RE: WWTP - TPS high level alarm
Hi Troy,
Just checking in on the proposal. I would also like to give you a call tomorrow around 8:30-9 AM for a few questions. Will you be available by phone?
Sincerely,
Jennifer Cline, P.E.
City of Molalla
Public Works Director
O: 503.759.0218
F: 503.829.3676
From: Troy Collison [mailto:tbc@industrialsystems-inc.com]
Sent: Monday, May 11, 2015 5:15 PM
To: Jennifer Cline
Cc: Dan Huff; Jason Clifford; John Sowards
Subject: RE: WWTP - TPS high level alarm
Jennifer,
You are most welcome for the assist today.
We are working up a proposal and should have it to you by end of day tomorrow!
Best regards,
Troy B. Collison
5835 NE 122nd Avenue, Suite #110
Portland, Oregon 97230
http://www.industrialsystems-inc.com
Phone: 503.262.0367
Cell: 360.957.5039
Fax: 503.262.0371
CCB # 196597 WA #INDUSSI880K9
AK# 1018436
From: Jennifer Cline [mailto:jcline@cityofmolalla.com]
Sent: Monday, May 11, 2015 4:53 PM
To: Troy Collison
Cc: Dan Huff; Jason Clifford
Subject: RE: WWTP - TPS high level alarm
Hi Troy,
Thank you and also for your prompt response time today to the plant.
At your earliest convenience, please provide quote to correct the solution you mentioned below.
Regards,
Jennifer Cline, P.E.
City of Molalla
Public Works Director
O: 503.759.0218
F: 503.829.3676
From: Troy Collison [mailto:tbc@industrialsystems-inc.com]
Sent: Monday, May 11, 2015 11:33 AM
To: Jennifer Cline
Cc: John Sowards
Subject: WWTP - TPS high level alarm
Hi Jennifer,
John Sowards from our office went to the site after your phone call this morning about the high level at the Transfer Pump Station (TPS) basin that did not cause an alarm to be dialed out.
I also connected remotely to the SCADA computer at this site and together we went through what was reported by the operators and what the alarm logs on the SCADA system show over the weekend.
We have determined the following:
1) The Transfer Pumps at the TPS were all disabled over the weekend so they could not run.
2) It appears that the communications between the TPS and the main PLC may have went to sleep due to inactivity and that comms were reestablished as soon as the operators attempted to make changes to the TPS control screen on the SCADA system.
3) Communications between the main PLC and other PLC’s such as the headworks appear to have remained fully functional as several alarms were reported from the headworks over the weekend.
4) The Ethernet Switch that communicates between the various PLC’s at the WWTP is approximately 10-15 years old and is an office type device with no ability to monitor the health of the ports.
If it is OK with you, we will replace this switch with an industrial thin managed switch and we will configure communications loss alarms between the main PLC and each of the remote PLCs to ensure such an event does not go unreported in the future.
Please call or email with any questions.
Thanks,
Troy B. Collison
5835 NE 122nd Avenue, Suite #110
Portland, Oregon 97230
http://www.industrialsystems-inc.com
Phone: 503.262.0367
Cell: 360.957.5039
Fax: 503.262.0371
CCB # 196597 WA #INDUSSI880K9
AK# 1018436